The Ulysses Spacecraft was sent to Jupiter to change it's orbital inclination via a gravity assist, and enter a nearly polar heliocentric orbit with an inclination of 79° in respect to the solar plane.
Entering an equatorial LEO in the direction of Earth's rotation requires about 9.4 km/s of $\Delta V$. The amount of $\Delta V$ to enter a Jupiter-Earth transfer orbit requires about 6 km/s from LEO. So I don't see a major problem from a Delta-V budget standpoint to enter a polar LEO, and then escape Earth's gravitational pull to enter a heliocentric orbit.
Question: Why didn't Ulysses, or any future spacecraft intended to study the poles of the Sun, enter a polar orbit around Earth, then fire it's rocket engine to obtain an inclined heliocentric orbit?
Note: Ulysses was launched aboard Space Shuttle Discovery, and the Space Shuttle couldn't enter a polar orbit, but let's ignore this issue as this question isn't specifically about Ulysses but about all polar heliocentric orbit satellites.