What engine is the most efficient in fuel and/or time to go from Earth orbit to Mars orbit and back again that are not capable of launching from the ground?
1 Answer
Fuel and time probably give different answers and it also depends on whether you want proved current hardware, engineering studies or wild speculation.
Let's maybe consider some options:
Chemical rockets a small rocket with a large fuel tank, using a high $I_{sp}$ fuel mix such as liquid hydrogen and liquid oxygen is an option. Freed of the need for enough thrust to get off the ground (or enough physical strength to stand up on Earth's surface) such a rocket can have a higher ratio of fuel to "dry mass" than a launcher and so achieve a higher delta-V. You need a lot of fuel, but it's relatively cheap fuel and (see any pork chop plot) you could hope to do the transfer in 3 months or so, aerobraking on arrival. You'd need facilities to make, store, handle and launch liquid hydrogen on Mars, which would take some work
Nuclear thermal rockets are a bit more fuel efficient than chemical rockets, while still offering high thrust. Some tests were done a few decades ago. For a number of reasons you might not want people operating them in LEO.
Ion engines are immensely fuel efficient, but provide very low thrust. A few tons of Xenon propellant will move quite a few tons of payload between these orbits, but it will take years. Xenon may be hard or impossible to source on Mars, but other propellants may work.
Solar sails are almost untested although extensively studied. They consume no fuel, but produce very little thrust. They can't be used in low Earth or Mars orbit, due to atmospheric drag.
Laser pushed lightsails are still more speculative, but could, in principle, achieve quite high thrusts for no fuel carried on the mission. They need large fixed infrastructure (lasers, and focusing lenses).
Nuclear pulse propulsion (project Orion) offers high thrust and relatively low mass of (rather specialised) propellant. This could move large payloads quite quickly between Earth and Mars orbits, but again you might not want it being used in, or below LEO. In theory you could launch from Earth with this technology, but I'd rather you didn't