The Voyagers will not deplete their fuel for centuries. Within a few more years, however, the rest of the involved system will degrade to the point that the voyagers can no longer transmit.
IR telescopes will be able to spot them, provided they don't have a course change, as the Plutonium in the RTG continues to decay, and it's decay products continue to decay, for many years, probably at least a century.
Wikipedia shows that the RTG has about 2400W of thermal power, and the reason the three RTG's are out on the boom was to prevent excess heating of the rest of the probe. Internally, the RTG's are about 1000° C, and cooling slowly. Given the half-life of 87 years for the Plutonium, it will still be about or above 400°C in the 2050's. The junction couplings are likely to corrode to non-function before then. The relatively high temperature will generate a strong IR signature via the principle of blackbody radiation. Still, it's only a 400W signature at launch.
Given the distance regime (101-124 AU) where it's at, the IR signature should be detectable for at least the next 50 years, and probably considerably longer, tho', thanks to the Inverse Square Law of EM Radiation, the IR signature is a tiny source. You'll need to be looking for it.
The distance regime also means that optical is not particularly useful. The light from the sun is too dim to be readily detected with current telescopes, tho' a hubble image might be doable with sufficient exposure time. At 101.8 and 124 AU, they receiver 1/10363 and 1/15376 the energy as an object near earth. And with only around 13 square meters of cross sectional surface area, most of which is angled, that's VERY little being reflected back.