Are all rocket launch sites surrounded by water bodies (such as sea)?
If so, what is the purpose of having a rocket launch site near a water body?
No, not surrounded by water at least. There are of course advantages to having a body of water nearby, for example in an event of a larger fire the fire fighting helicopters and airplanes would have a shorter flight between the fire and where they scoop the water from. But that doesn't necessitate the water body completely surrounding the launch site. In fact, there are many launch sites that don't have their entire perimeter surrounded by water, for example the Baikonur Cosmodrome in Kazakhstan (which does have a large Syr Darya river relatively close by, but is not surrounded by it):
Aerial photograph of the Baikonur Cosmodrome launch site area (source: Google Maps)
There is a clear need though to have a wide degree arc of land oriented towards the East that is unpopulated, in case of an in-air launch failure, and to have easy access to first rocket stages and boosters falling / landing back to Earth. That is due to the Earth's rotation, and for the same reason the rockets that are launched appear to have a radial path moving away from the launch site towards the East.
If it's better having such area mostly of water surfaces, or merely unpopulated land, is however debatable and depends on launch operations' ability to retrieve objects that have fallen back to Earth. Obviously, you wouldn't want a large forest area, to avoid fires caused by falling debris and rocket stages, in case their landing parachute didn't deploy properly. You'd also want easy access to all areas of this arc, which might prove easier on water, than on land, if there is not enough access roads, the land is overgrown, marshy, or any other reason why you wouldn't be able to access sites of fallen debris easy.
The question may be re-phrased to read - Where should a space launch site be located?
Just like a car with an IC engine, a spacecraft is really riding a controlled explosion. Therefore it makes sense to reduce the risk at launch - hence, distant from centres of population. An example of what happens when control is lost may be viewed here A water body such as a sea/ocean is a preference, but there are also space stations located elsewhere E.g. in a desert.
A list of active launch sites is available here For instance, Baikonur & Jiquan are situated in the interior
Desirable qualities of a launch site:
NASA (Florida) and ESA (French Guiana) fit the pattern. Sea Launch was designed with these factors in mind.