Leaving aside the very valid political elements of the Orion design choices.
As noted in the question, many costs/weights in manned space flight are fixed per craft regardless of crew size. A computer,antenna or sensor does not know how many crew there are and weigh the same. Some are also impact by square cube law, so doubling the internal space of a capsule does not double the mass of the hull surface area to contain it.
The actual consumables mass for adding additional humans is also relatively low element of overall mass, only a couple of kg per person per day and easy to turn into a double length mission for half the crew, much more easily than doubling endurance of a single seater.
The rocket to carry the capsule also do not directly scale, so building an engine twice as big does not necessarily make it twice as expensive (though having enough engines to survive a failure is nice), and many launch/mission costs are fixed or nearly so regardless of size at least up to the point where building/moving the thing around on the ground is a problem.
In practice there seems to be a sweet spot for rocket design smaller than Apollo but larger than Titan/Gemini where materials humans have access to intersect with square cube law to make reasonably efficient rockets per kg to orbit. And if you stick a single person capsule on that size rocket it starts to look a bit wasteful, and easier to scale up the capsule than scale down the rest of existing rocket infrastructure.
All of this means that making a smaller manned rocket is not necessarily much cheaper to design.
Redundancy/safety is also much easier to add to a multi crew vehicle, for example one spare set of life support fixtures/plumbing etc in a six person capsule is a much lower% of total mass than in a single person capsule. The biggest redundancy boost is in the crew themselves. The 2 person Apollo crew size was locked in very early in the progam, in large part based on the increased safety in having another set if hands in event of injuries/illness.
In addition much of the cost of a crewed rocket is in man rating. A new vehicle regardless of crew size would need to fly at least once empty, plus do a number of abort tests in various shapes (plus lots of man hours in contingency planning/modeling) so flying two single crew moon missions might involve building and destroying a dozen in tests before being human rated.
So even if not ideal for a given lunar mission a 6 person capsule that has flexibility for other missions may turn out cheaper in the long term assuming sensible choices.
It is of course possible to overdo it, with feature creep being the death of many many tech projects, so possibly making return to moon a single seater would have meant it flew years ago.
