I am making an exoplanet for a game and was wondering whether a planet without an atmosphere (or at least with a very weak atmosphere) could support caves that do have an atmosphere, or whether the air would escape through all the cracks immediately.
The caves are connected to the surface with some small holes. They also have liquid water and black smokers.
Yes, in fact it can happen. It happens on Earth to some extent. The term is a "Gas Pocket", and is most commonly associated with oil and natural gas deposits. There are other times of gasses that are sometimes stored as well under the Earth.
If there is a connection with the outside world in the form of a small hole, then the likelihood of there still being any gas in it is much lower. The only way this would work is if the gas was very heavy. Again, this happens to some extend on Earth, most notably in Radon being concentrated in some basements. This likely wouldn't lead to a thick atmosphere by any means, but could produce something.
The only way that it could sustain long term an atmosphere is if it was completely sealed up, or if there was a source generating the atmosphere on a regular basis. There are a few things that could cause this to happen. It could come from outgassing due to heat, from some kind of biological process, or possibly from some subterranean vents, which is really just an extension of the outgassing problem. It could also be caused by some kind of chemical reaction.