While the launch escape system (LES) is important for getting the crew away on ascent there are other thing required for human-rating a craft/launch system (and you do need to pay attention to the entire system and mission, not just the rocket during ascent).
Specifically for Atlas V they needed to: space.com
- upgrade the emergency detection system (your LES does no good if it doesn't get enough warning to get away in time)
- figure out how to get astronauts onto the rocket, including options for emergency evacuation
- develop a different upper stage (no explanation provided on this one)
NASA actually has a document available online that details what all goes into human-rating a spacecraft by their standards: Human-Rating Requirements for Space Systems
It starts off with the very basics:
The space system shall provide the capability to sustain a safe, habitable environment for the crew
Sets expectations for chance of failure:
The space system shall meet probabilistic safety criteria derived from the Agency-level safety goals and safety thresholds with a specified degree of certainty
Insists allowances be made for the fact that humans are operating the craft:
The space system shall be designed to tolerate inadvertent operator action (minimum of one inadvertent action), as identified by the human error analysis, without causing a catastrophic event
There's also an entire section on requirements for the launch escape system (LES). First, there has to be one:
The space system shall provide abort capability from the launch pad until Earth-orbit insertion to protect for the following ascent failure scenarios: ...
And beyond that, it has to be activatable by the crew, the ground control, and the rocket itself (only quoting the last):
The crewed space system shall monitor the Earth ascent launch vehicle performance and automatically initiate an abort when an impending catastrophic failure is detected
And at end of mission:
The space system shall provide recovery forces with the location of the spacecraft after return to Earth
tl;dr The launch escape system is a key requirement but it's intended to be a last-resort, not the only option for crew protection. You also need provisions for supporting a crew on the ground and in space.