Convective heat transfer within tenuous atmospheres wouldn't work, lunar exosphere is near vacuum, and it would be fairly limited on Mars with its average of ~ 0.6% mean sea-level Earth's atmospheric pressure, so yes, ICE blocks would have to be redesigned to either facilitate fuel and oxidizer also as a closed-loop liquid coolants (and also preheat them in the process to improve combustion), or use separate coolants (e.g. there plenty of dry ice precipitate in and above Martian regolith, so that's a lot of solid form CO2). Whichever option would be used, these coolants would be transporting excess heat into likely large and heavy radiators, losing heat predominantly via thermal radiation alone. But since you need to pump oxidizer to the ICE on top of fuel, they would have to be redesigned anyway.
So designs might be substantially different to what we're used to here on Earth, possibly depending on how you store your oxidizer (cryogenic LOX?), how much heat you're producing, and where else in the system you might want to use it. Mars obviously has an advantage in cooling ICE blocks with still some non-negligible atmospheric pressure and an average temperature at ~ -55°C, but that might also mean that you'd have to first heat the system up before even starting it. Excess heat could as well be redirected to heat up the cabin space, or otherwise made useful. But designs would be completely different for each celestial, even just local environment on any of them. For example, on Titan, you'd really only require the oxidizer, since there's plenty of methane in its lower atmosphere. See this answer for more details on scavenging design of such an engine.
Our ICE designs would adapt and evolve, like they have here on Earth for all kinds of environments, from dry and hot deserts, to arctic conditions. How? Well, I expect lots of new innovations, and some of them will be written in big letters in the history of any of the new worlds we'd colonize. And these innovation processes have already started, for example, Wickman Spacecraft & Propulsion Co. (WSPC) developed a way to directly burn Martian atmospheric CO2 with their Mars Jet Engine. Not an ICE, but there might be others with goals of developing an air-breather ICE suitable for Mars. The race to win the best design has barely started. Would they use CH4 + O2, Trisilane Si3H8 + CO2, something else entirely? Who knows ...