The relevant quantity to consider here is the specific orbital energy:

The specific orbital energy $\epsilon$ (or vis-viva energy) of two orbiting bodies is the constant sum of their mutual potential energy ($\epsilon_p$) and their total kinetic energy ($\epsilon_k$), divided by the reduced mass.

$$\begin{align} \epsilon &= \epsilon_k + \epsilon_p\\ &= \frac{v^2}{2} - \frac{\mu}{r} = -\frac{\mu}{2a} \end{align}$$

where

• $v$ is the relative orbital speed;
• $r$ is the orbital distance between the bodies;
• $\mu = G(m_1+m_2)$ is the sum of the standard gravitational parameters of the bodies;
• $a$ is the semi-major axis.

For an elliptic orbit the specific orbital energy is the negative of the additional energy required to accelerate a mass of one kilogram to escape velocity (parabolic orbit).

A body that's gravitationally bound has $\epsilon<0$ and if you increase its specific orbital energy to zero it will become unbound and either escape the system or fall into the primary, depending on the direction it's heading.

For a circular orbit, $r = a$, and $\epsilon_k$ and $\epsilon_p$ are both constant. So $$\epsilon = \epsilon_p/2 = -\epsilon_k$$

Rearranging the earlier equation, we get the celebrated vis-viva equation: $$v^2 = \mu\left(\frac{2}{r} -\frac{1}{a}\right)$$

We can use this equation to calculate the speed changes necessary to move from one circular orbit to another. The simplest way to do that is the Hohmann transfer orbit. It's also the transfer which has the minimum energy requirements. (In the real world of non-circular, non-Keplerian orbits, there can be lower energy solutions). The Hohmann transfer uses (half of) an elliptic trajectory with a major axis equal to the sum of the radii of the two circular orbits:

At the start of the manoeuvre, you do a tangential burn which changes the orbital speed and hence the orbital energy. At the other end, you do another tangential burn to circularize the orbit... unless you want to loop back to your starting point.

To drop from the orbit of Neptune (~30 au) to Earth orbit, you need to shed most of your orbital speed. That's not a major ordeal because mean orbital speed is fairly sedate at that distance. However, when you get to 1 au your speed will be huge, and it takes a lot of delta-vee to circularize your orbit.

It might be more intuitive to consider the reverse manoeuvre; the energy requirements are identical because gravitation is a conservative force. To send a body from Earth orbit to Neptune takes a lot of energy. Your almost kicking it out of the system, and then there's the additional energy needed to circularize the orbit when you reach your destination.

Here are the relevant numbers, calculated in Python, mostly using the vis-viva equation.

Here's a live version of the Python script, running on the SageMathCell server.

At the start of the transfer, you have to shed ~75% of your current orbital speed. And when you reach 1 au, your speed is almost 40% higher than Earth's so you have a lot of kinetic energy you need to shed, otherwise you'll loop back to Neptune.