Imagine your telescope optics looked like this red-hot glass!

Herschel's instruments look at the world in the wavelength range of 55–672 µm. When plotted as a function of wavelength, the thermal spectrum of a black body peaks $ \approx 5 k_\mathrm B T$. The boiling point of liquid helium is 4.2 K. The peak wavelength for something at that temperature would then be given by
$$\lambda_\mathrm{max} = \frac{h c}{5 k_\mathrm B T}.$$
With Boltzman constant $k_\mathrm B$ of about 1.381E-23 J/K and the Planck's constant of 6.626E-34 J s, the spectrum should peak at about 700 µm, so even if the optics were at liquid helium temperature it would be hard to see faint objects at longer wavelengths above the brightly glowing mirrors and optics.
According to Wikipedia:
The light reflected by the mirror was focused onto three instruments, whose detectors were kept at temperatures below 2 K (−271 °C). The instruments were cooled with over 2,300 litres (510 imp gal; 610 US gal) of liquid helium, boiling away in a near vacuum at a temperature of approximately 1.4 K (−272 °C). The supply of helium on board the spacecraft was a fundamental limit to the operational lifetime of the space observatory; it was originally expected to be operational for at least three years.
So they let the helium boil at very low pressure in order to bring its boiling point down to 1.4 K.
The main mirror was highly reflective in the infrared, which means its emissivity was low, probably down near 0.01. This helps reduce the emission even farther. According to the BBC's Herschel space telescope finishes mission the primary mirror was at about 90 K
For at least one of the instruments SPIRE internal mirrors and the sensors needed to be kept at 0.3 K. The refrigerator for that needed a cooling mass on its "hot side" and a slow boil-off of liquid helium provided such a sink.

The Herschel telescope had to be kept extremely cold to study its frigid targets
Okay Cold, but Why Helium and not the Cold of Space?
Cooling to space is limited by the Cosmic microwave background, and its characteristic temperature is about 2.7 K. That's just not cold enough for the telescope's optics.
Also, while radiating into the cold of space theoretically provides a source for cooling besides liquid helium, this is pretty inefficient and all it takes is a short exposure to the hot Earth or a tiny bit of Sunlight to warm everything up dramatically, rendering Herschel at least temporarily blind.
Instead, with the current design, the cold optical system can be boxed in and carefully insulated, and the boil-off of the Helium vented to space carries the heat away.
Further reading:

Herschel pictured the "cold cosmos" - places where gas and dust are coming together to form stars. Here, in the Rosette Nebula, in the constellation of Monoceros, a mass of new stars (bright spots) are just firing into life