How safe a space settlement is depends on how well-established it is. The very first lunar base using inflatable structures would be vulnerable to radiation and small meteors as well as a critical malfunction, such as oxygen storage.
A well established base would probably be underground, at the pole, and have it’s own repair and refining capability. In time they would develop the ability to refine oxygen, water, aluminum, silicone and other local resources. The evolution from a temporary base to a self-sustaining colony could happen over time. The goal would be to be able to supply more needs locally and import less from the earth every year. That self-sufficiency can't happen in Earth orbit because there are no local resources. With self-sufficiency comes safety. Instead of fragile, lightweight, high-performance objects lifted from the earth, the structure of the colony and everything in it could be more massive and heavy duty, if it were made from lunar materials.
A well established lunar colony would need a completely reusable travel infrastructure. This would best be imagined in three legs, with completely reusable spacecraft. The first leg to low Earth orbit. The second leg would be and electric or nuclear rocket going from Earth orbit to lunar orbit. Finally would be a lunar lander going from the lunar surface to lunar orbit. This approach would not require a huge delta Vee for any of the rockets. That way they could be built robustly, to be practically reusable. So the Apollo model just doesn't apply.
A lunar settlement would be more vulnerable at first, but much safer in the long run.