Consider the following hypothetical event: a Voyager spacecraft (1 or 2) changes its planned interstellar trajectory quite suddenly (I'm well aware of motion and gravity laws and I know that this shouldn't occur, but let's just pretend they are flying by an unobserved massive object). Would the engineer team here on Earth detect it? How?
I've read a lot through Voyager Interstellar Mission website, but I couldn't find any detailed information on how spacecraft position and speed is determined.
Would the engineer team here on Earth detect it? How?
There are two issues here.
Suppose Voyager 2 (Voyager 1 is moving faster and is further away, so the effect will be lesser on Voyager 1 than on Voyager 2) made a ridiculously close flyby of an object with a similar size and mass of Neptune. Such a flyby would result in a delta-V of about 8.2 km/sec. Suppose this delta-V is orthogonal to the line of sight to Earth (worst case). Even then, it would be almost 41 days to fly out of the beam width of the Deep Space Network antennae. To this day, JPL still receives data from the Voyagers on a daily basis. That gives plenty of time to notice the change in velocity. It appears that continued contact with the Voyagers could continue even in the face of a ridiculously close flyby of a ridiculously large object.
However, such a ridiculously close flyby would have another effect. We don't "see" the Voyager satellites. We instead receive their transmissions. The vehicles are invisible if we can't receive those transmissions.
The problem is that a flyby not only changes the satellite's velocity, it also changes the satellite's orientation and rotation rate thanks to gravity gradient torque. That's fine in the case of a planned flyby. It's no so fine in the case of an unplanned flyby when the vehicle has limited fuel to correct for an unplanned rotation and limited intelligence to orient itself so the antenna points back to Earth.
The only downlink capability currently available to Voyagers is the X-band transmission, with a beam width of 0.5 degrees. Even a small unplanned change in Voyager attitude means we on Earth couldn't see that Voyager, at least not until the Voyager has found out where it is pointing and corrected for that. A change in attitude rate is an even bigger problem. The Voyagers have a very limited amount of remaining fuel for attitude control. While that limited amount of fuel is more than enough to last until 2025 assuming that the vehicles are flying through empty space, whether it's enough to correct for an unexpected large gravity gradient torque is a very different matter.
Spacecraft position and speed are determined from Earth: we point an antenna at the point where we expect the spacecraft to be, and we try to receive its radio signal. The antenna position that gives the strongest signal gives a heading.
To determine distance, we send a signal to the spacecraft and we wait for the response.
You see the problem: an unexpected change in trajectory means the spacecraft is no longer in the position we expect. We point the antenna in the wrong direction, and we don't find the spacecraft.
A search is possible (gradually sweep the antenna across the sky), but would take a long time.
There are several different techniques to measure the speed and position of a spacecraft. First, distance from Earth can be measured by the round-trip time of a signal. If the time the spacecraft needs between receiving a signal and sending it back is well known, distance can be determined very precisely. From two measurements of distance, one can calculate the speed of the vehicle This is, the rate it moves away from Earth, not the total speed. There is a second method for calculating this speed by observing the frequency of the received signal: The faster the spacecraft moves away from Earth, the lower the frequency gets (see Doppler shift). These measurements can have almost arbitrary precision, only limited by the length of observation. A good example is the observation of probes doing a fly-by of Earth whose speed has been measured to less than 1 mm/s precision.
The precise position in space, on the other hand, is difficult to measure. As Hobbes wrote, one can use a directional antenna and do a scan to find the best position. But, even the 70 meter antennas of the Deep Space Network have a beam width of 0.1 degree. We can assume that we can, through repeated measurements and interpolation with this antenna, determine the direction of the spacecraft with a precision of 0.01 degrees. In the case of Voyager one, 130 AU from Earth this results in a position error of 3 million kilometers. But, this large opening angle of the beam sent by the antenna also has the advantage that the spacecraft can not easily vanish from the space covered by the transmission due to some unexpected change in trajectory.
In summary, changes in speed and distance from Earth can be determined very precisely, the absolute position in lateral coordinates is a much trickier task and difficult to find changes here.
On top of the mentioned "roundtrip" and "antena aim" data, which is good for rough measurements (not very rough, but not exactly a sub-meter precision), there's a way to triangulate the position of any broadcasting object.
Point three antennas at different locations of the world, with precisely synchronized timers, at Voyager. Record the incoming wave, precisely recording the timing of some significant point.
The X-band wavelength is about 3cm. Measuring speed is quite simple, calculate deviation from that wavelength, caused by "red shift". But moreover, if you compare phase of the wave - timings of arrival of a specific point of it - you can triangulate the craft's position with extreme accuracy, way better than "signal strength" estimate.
