Your question specifically asks about "reliable" thrusters, but makes no mention of any other criteria (such as whether it be used for interplanetary missions). I will focus this answer on reliability, since cost is too complicated and mission-dependent. In this case, we should look for simplicity above all else. Simpler may mean more reliable and in that case a cold gas thruster probably fits that bill. It's just a tank, a valve, and a nozzle.
The engine that probably fits the bill for "minimal reliable engine" that has a chance of interplanetary flight would be the Apollo LM ascent stage engine. Simply put, this engine had to work no matter what - if the descent stage engine failed during descent, it had to be able to abort the landing, and it was the only engine that they had to lift off the moon.
The design was simplified with the intention of increasing reliability:
- Pressure fed (why bother with fuel pumps?)
- Had no throttle (why bother with throttle?)
- Non-gimballed (why bother with the extra complexity?)
At that point it wasn't much more than a combustion chamber and a nozzle (well...)
So let's break this down into what is usually meant by "reliable": a rocket engine that is capable of long-duration spaceflight. Where this spaceflight is taking place is not relevant - LEO, Lunar orbit, interplanetary etc. Let's further require that if our engine fails then the mission will be catastrophically affected. What I mean here is basically human, long-range spaceflight, where loss of the engine means loss of the crew, and where the engine has to undergo long periods of being off, and where the engine has to fire again to get home. I am further assuming that we are looking only at technologies that have been tried in space. This more or less restricts us to chemical engines, since other types of engines have not been tried on human spaceflight.
Magically, NASA has done exactly this study (read more here). If we extend the definition of "rocket engine" to mean "multiple independent engines in a single stage", which can increase reliability at the expense of complexity, NASAs big ones are below, but there are more in that document:
Number of engines on the stage - in theory more is better up to a point where complexity and single engine failure rates take over. As we saw on at least Apollo 13, failure of one engine in a 5 engine stage was not a problem
Operation duration - in theory, engines that operate for less time are more reliable as they have less time to fail. However, if delta-V requirements for the mission is high it may be more reliable to fly a smaller engine for longer.
Engine thrust - in the study I reference, they have yet to do the study, but expect that lower thrust engines are more reliable. In fact, the study states that 90% of failures are correlated with thrust level alone (!)
Engine re-start - an engine is less reliable the more it has to start and stop
Here is the full diagram of what affects engine reliability. I'm afraid that it's more complex than an SE answer can give, so read the document!