The biggest difference is the nozzle. For optimal performance in vacuum, you want a much larger one.
According to Spaceflight 101, the chamber pressure is the same, but the expansion ratio (throat area to end-of-nozzle area) is 7 times larger in the vacuum variant, which (if correct) implies about 2.7 times the nozzle diameter if the throat is unchanged.
The Wikipedia description of the 1C-vacuum says the expansion nozzle length is 2.7 meters, while the overall length of the first-stage 1C is only 2.9 meters long - roughly half of that being nozzle. So the nozzle length is basically doubled. Presumably the relationship between the 1D and 1D vacuum is analogous.
This pic is said to be, left to right: Falcon 1 Merlin 1C, Falcon 9 1C (different mounting), and Falcon 9 2nd stage 1C vacuum -- without the extension nozzle, so it's a shorter, fatter nozzle than the others.
And here's what the extension nozzle looks like by itself:
Since the Falcon 9 second stage mounts a single engine, in the same diameter body as the first stage (with its cluster of 9), there's plenty of room for the large nozzle.
This reddit thread includes some inconclusive debate about how much different the vacuum engines actually are. There are certainly differences in mounting and layout (most obviously the gas generator exhaust nozzle is canted further out to avoid impinging on the nozzle extension), but the turbopumps etc. are apparently the same.
According to the October 2015 revision of the Falcon 9 user's guide, the 1D Vacuum has a much deeper throttling capability than the first-stage engine, down to about 40% of maximum thrust (360kN-934kN). It's unclear what the reason for the throttling difference is.