I would like to make my own compilation that includes several missions not mentioned in any other answer so far. (The record is actually 483 km of Soyuz TMA-1, not the 475 km of Soyuz TMA-11.)
Gemini 5: splashed down 130 km off course due to programming error. Someone put in the rotation rate of Earth for the solar day instead of the sidereal day.
Voskhod 2: landed 382 km off course due to a delay caused by malfunction of the automatic re-entry system. This is probably the most dangerous off-course re-entry for several reasons. It came down over forest (Voskhod had braking rockets under its parachutes, triggered by a contact string hanging under the capsule, but fortunately the string did not hit a tree early). The capsule got wedged between two trees and was stuck in Siberian forest with bears, wolves, and cold weather for a day and a night. Helicopters at that time apparently could not lift them out nor land nearby due to heavy forest, so the crew had to ski out.
Mercury-Atlas 7: splashed down 400 km off course due to delays from pilot error.
Soyuz 5 had a ballistic re-entry due to failure to separate from the Orbital Module. I could not find how far off course it landed. This YouTube thing says "over 2,000 km" off course but I find that physically impossible as the other Soyuz missions suffering similar separation problems did not exceed 500 km off course. Spacefacts.de says "hundreds of kilometers short". Astronautix only says "far short of its aim point".
(Soyuz 5 landed very rough because of partial tangling of chute lines and failure of brake rockets. Cosmonaut Volynov broke several teeth from this, so it may be tied as the most dangerous off-course landing.)
Soyuz TMA-1 landed 483 km off course according to Spacefacts.de. It had a ballistic re-entry due to failure to separate from the Orbital Module.
Soyuz TMA-10 had a ballistic re-entry for the same reason and landed about 304 km off course.
Soyuz TMA-11: landed 475 km off course according to Wikipedia, and 428 km according to Spacefacts.de. Astronautix says "They landed 470 km short of the target point". The Wiki entry has no cite for its number. It is unclear how Spacefacts and Astronautix came up with their numbers. Soyuz TMA-11 suffered a ballistic re-entry for the same reasons as Soyuz TMA-10.
(There is also Soyuz 7K-T No. 39, also known as Soyuz 18a. It never made it to orbit, but it did go above 145 km and thus into space, so I'm not sure if we should count it. A failure in separating stage 1 (core) from stage 2 (highest stage) caused the rocket to go far off course, and the launch was aborted. The capsule came down over 1,000 km from the launch site, though I don't know how far away it was from the intended landing site.)
…Speaking of more separation failures…
Vostok 1, Vostok 2, and Vostok 5 all failed to separate their capsules from the service modules at the beginning of re-entry. Vostok 1 (the very first manned mission in space) landed 280 km off course. I could not find how far Vostok 2 and Vostok 5 landed off course. Is it possible that one of them went farther than 483 km? I don't know. But note that, in all Vostok missions, the lone pilot ejected before the capsule hit the ground. Both landed under parachute and it is not clear whether the Vostok 1 capsule or Yuri Gagarin landed 280 km from the planned site, nor is it clear how far apart the capsule landed from the pilot for any of those Vostok missions.
So for crewed missions that actually reached orbit and landed or splashed down intact, it is Soyuz TMA-1 (not 11) that landed the farthest off course… so far. There is still a chance that Soyuz 5 landed even farther off course, since I could not find the exact number.