Everything I know about orbital mechanics I know from Kerbal Space Program, so this may be a dumb question.
Using (possibly wrong) math, it looks like that even for the lowest delta-v (to Earth) objects we know about, with existing chemical rockets we probably couldn't bring back a fragment bigger than a few tons, at least within a reasonable number of orbits. But with a nuclear rocket, it might not only be possible, but pretty easy.
After playing KSP, I got this very off the wall idea, that Chelyabinsk - possibly a piece of (86039) 1999 NC43 - could have been a weapons test. It came from the direction of the sun, which is exactly what I'd do if I wanted to throw a rock at someone.
You could argue that it would be more efficient, in terms of spent energy, if you just launched a nuke directly. But kinetic bombardment has two major advantages: 1. no fallout, 2. total deniability.
What is the current consensus about the feasibility of such a weapon? At what point, if ever, should we worry about this?