If that's the case, then an accelerometer on Earth's surface would always read "acceleration" of 9.8m/s^2 in vertical direction due to Earth's gravity. Is my train of thought correct, or am I completely out to lunch?
You are correct, but only to two decimal places of accuracy. An accelerometer that gives only two decimal places of accuracy is an extremely lousy accelerometer.
At sea level and at the equator, acceleration due to gravity is 9.8142 m/s$^2$, directed toward the center of the Earth. An accelerometer will register an acceleration of only 9.7803 m/s$^2$, directed upward. These differ in magnitude by 0.0339 m/s$^2$, or about 0.34%. (Note well: They differ in direction by 180 degrees.) Even a lousy accelerometer is good to three places of accuracy, so this variation of 0.34% is within the realm of detection by a mediocre accelerometer (but not an extremely lousy accelerometer).
From a Newtonian perspective, an ideal accelerometer measures the acceleration due to the net sum of all real forces except gravity. What about fictitious forces such as the centrifugal and Coriolis forces? They don't count from a Newtonian perspective. They're fictitious forces. Ideal accelerometers don't sense those observer-dependent fictitious forces. What about gravity (which is a real force in Newtonian mechanics)? Accelerometers don't detect that, either. That's the only real force accelerometers don't detect from a Newtonian perspective.
From a relativistic perspective, an ideal accelerometer measures proper acceleration. Gravitation is a fictitious force in general relativity. A simple way to put this: An ideal accelerometer measures the acceleration due to the net sum of all real forces, period. Another way to put this: accelerometers measure all real forces except gravitation (the same explanation as the Newtonian explanation).
There are some subtleties that the above ignores. @Rikki-Tikki-Tavi mentioned frame dragging in a comment. Unless you are orbiting a rotating black hole, frame dragging is exceptionally small. One would need an extremely sensitive accelerometer to sense frame dragging, and even then only after a whole lot of analysis. Gravity Probe B was intended to measure those relativistic effects. This experiment was at best a partial success. The subtleties to which @Rikki-Tikki-Tavi alluded are very, very small for the Earth.
Now, if I wanted to devise a very precise INS (inertial navigation system) for interplanetary travel, I would need to constantly account for "false acceleration" due to gravity of nearby (or all?) planets and their moons within the solar system. Is that right?
Since accelerometers do not detect gravitation, deducing the evolution of a spacecrafts state (position and velocity) means that a spacecraft needs decent gravity models of the objects that affect the spacecraft gravitationally. This is called deduced reckoning, or dead reckoning for short.
There's a good reason for the change from ded(uced) to dead. A spacecraft that only relies on dead reckoning alone is soon to be dead.
I mentioned "ideal accelerometers" multiple times above. Real accelerometers differ from the ideal in a number of ways. A real accelerometer might report an acceleration of 1.001 m/s$^2$ when it should have reported an acceleration of 1 m/s$^2$, and an acceleration of 2.002 m/s$^2$ when it should have reported an acceleration of 2 m/s$^2$. This is a scale factor error.
Take an accelerometer apart and you'll see multiple devices that ideally measure acceleration in three orthogonal directions. In practice, they don't. Real accelerometers have a non-orthogonality error.
Scale factor and non-orthogonality are examples of the systematic errors associated with an accelerometer. Systematic errors can be addressed. A decent navigation system will try to estimate these systematic errors.
What can't be addressed are random errors. Accelerometers are inherently noisy devices, ideally white noise. Integrated white noise results in a random walk. The deduced velocity vector takes a random walk from the true velocity vector. Flight software computes position by integrating velocity. This results in an integrated random walk error.
Over time, this can result in very bad estimates of position. That's why dead reckoning means you are dead. The only way to overcome this is to have an alternate measure of position. GPS does a very nice job for vehicles in low Earth orbit. Something else is needed for vehicles beyond LEO.