Your question indicates to me that although you aren't familiar with celestial mechanics, you do have some knowledge of physics and astronomy. You're a space enthusiast? *Good for you!!*

The situation you describe is very much aligned with what we call *[Keplerian orbits][1]*, named for their pioneer, Johannes Kepler, publishing in the early 1600's.

In the absence of perturbations by other gravitating bodies, which I assume comes under your prohibition on "obscure funky disturbances", *Keplerian orbits are indeed stable.*

Where did your reasoning lead you astray? Assume the orbiting object is in a circular orbit, as shown by the dark line in the figure, orbiting in the direction shown. At the large red arrow you gave the orbiting object an instantaneous gentle nudge inward, indeed not changing the tangential velocity at that point.

[![A circular Keplerian orbit (dark green) around a spherically symmetrical primary (blue), and the orbit after a vertically downward perturbation at the large red arrow][2]][2]

But you did change, even if only a little, the *flight path angle*, to that indicated by the small red arrow on the red orbit. The object is no longer traveling horizontally, it's headed a bit downhill. When the object was traveling strictly horizontally (circular orbit) the local gravity vector was perpendicular to the velocity vector. When those vectors are perpendicular there is no change in the object's speed. (*Speed* is a scalar quantity, the magnitude of *velocity*, which is a vector quantity with both speed *and direction*) In the downhill case, post-nudge, just as in the case of something rolling down a hill on Earth, there's a (small) component of the gravity vector parallel to the velocity vector. The object *accelerates*, i.e. its speed increases with time, so *its speed doesn't remain constant after the perturbation*. That change in speed corresponds to the change in gravitational potential energy due to the changing radius from the primary's center: the farther downward it goes, the faster it goes.

This increase in speed with decreasing altitude causes the orbit's radius of curvature to be larger than that of a circular orbit at that altitude, so the orbit eventually bottoms out at the *periapsis*, 90° away (as measured from the center of the primary) from the perturbation point. It then rises back to the original altitude, 180° away from the perturbation, with the same tangential speed and the same vertical speed, just upward instead of downward, as you see at the bottom of the diagram.

That vertical speed carries the object higher, and that decelerates it. The reduced speed decreases the orbit's curvature, so it peaks out at *apoapsis* 270° from the perturbation and begins descending. At 360° from the perturbation — one orbit — it is back to exactly where it started, at the instant of the perturbation, with the same velocity, same flight path angle, same everything, and this repeats *ad infinitum*.

This orbit, like all bound (i.e., not escaping) Keplerian orbits, is perfectly stable. Given the constraints you listed, it would remain exactly as shown forever, without any kind of control.

If you made the perturbation *not* small, say a significant fraction of the orbit speed, then you *could* make the object collide with the primary. "If you push something hard enough, it will fall over."

Once you start complicating the picture — the planet isn't spherically symmetrical, the planet rotates, there are other gravitating bodies involved, part of the orbit is in full sunlight and part is eclipsed, etc. — then those perturbations make the orbit evolve (change with time), in a few cases to the point of colliding with the primary or even being ejected from the system. Orbit evolution happens to everything orbiting Earth, even the [moon][3].


  [1]: https://en.wikipedia.org/wiki/Kepler_orbit
  [2]: https://i.sstatic.net/lNnTC.jpg
  [3]: https://en.wikipedia.org/wiki/Orbit_of_the_Moon