There are tradeoffs at work here.
The current configuration allows them to use one first stage design for 1-5 boosters. A first stage that has symmetric attachment points, needs 2+4+4 = 10 attachment points for the 2, 3, 4, and 5 booster cases.
Another approach would be to discard the odd numbered configurations altogether and go to 2, 4, 6, 8 boosters. That means a higher manufacturing cost and more dead weight (more attachment points, less efficient boosters).
In both cases there's a tradeoff between manufacturing cost and complexity, and rocket efficiency (performance loss due to non-optimal configurations). Apparently the tradeoff was favorable for the current 1-5 booster configuration.
The Atlas V first stage is built to in a single configuration. Here's the first stage for OSIRIS-REx, an Atlas V 441 launch with one booster attached. I've indicated attachment points for 2 boosters, indicating they will fly with unused attachment points.
This photo also shows there's lots of places around the circumference of the stage where you can't attach boosters due to LOX lines or electronics attached to the outside. This is why e.g. the 4-booster variant is asymmetrical. Apparently, the Common Core Booster was not initially designed to be used with solid boosters, and when they had to be added later they didn't want to rearrange the rest of the stage.
Being designed for government contracts in an era with very little competition, it's feasible they didn't want to spend money wringing out the last bit of performance from the design.