With SpaceX creating their launch facility in Boca Chica Texas, right next to the water, why would they bother building a huge launch pad to withstand the blast, as opposed to building it on top of the water? Why not let 20 feet of water handle the heat and blast? It would need a very good support structure below, but an unlimited water supply might make the job worth it.
Salt does all sorts of unpleasant things to just about every building material humans use. Hot salt spray, such as you'd get from a rocket launch, is even worse: spraying something with hot saltwater is one of the techniques used for corrosion testing. Build a launch pad over the ocean, and you'll need to clean it off after each launch to try to keep the corrosion down to reasonable levels.
Even without the rocket launches, building something to survive the ocean is difficult. Concrete, for example, is vulnerable to haloclastic weathering, where salt crystals deposited by evaporating water split it into fine sand. Steel, even stainless steel, will rust when exposed to saltwater.
Building a launch pad to withstand the blast of a rocket launch is simple compared to building it to withstand the ocean.
Environmental impact may be a major consideration
The effect on the local water environment from that sort of blast into it could be severe and far reaching in the neighborhood around a water based launch pad.
Also if there are any abort or need to dump fuel or uncombusted fuels from explosions could also have massive impacts that are immediately spread due to being in open water.
Where as on land you can tightly contain such effects to a much smaller area that can be cleaned up and/or doesn't spread easily and immediately through water (long term is different of course due to groundwater contamination, etc.).
Per the answers here the water around rocket launches is in spray or mist form to absorb energy through evaporation, with cooling being largely a happy side effect. A flat sea surface will instead tend to reflect energy back towards the launch structure and rocket. The rocket blast will tend to displace it downwards but not actually absorb the much energy So the over water platform is probably worse than a conventional 45 degree blast deflector that sends the exhaust sideways to dissipate.
Being at sea does have some other potential uses, in terms of fewer neighbors to get upset when launches fail, and if you make your full launch structure mobile there are potential commercial advantages but as per Marks answer, generally adding the ocean to any plan involves dealing with the fact it breaks everything