The J-2 engine used on the upper stages of the Saturn launchers has a specific impulse in vacuum of 421 seconds. This is substantially lower than that of the smaller RL10, 440-460 seconds depending on the model. The RL10 has a greater nozzle expansion ratio -- 47:1 up to 280:1 depending on the model, compared to 27:1 for J-2.
While the 5 J-2 engines were fairly crowded in the base of the Saturn V second stage, the single engine in the third stage is not; a nozzle extension doubling its 2.1 m diameter to 4.2 m would still allow for reasonable clearance versus the 6.6 m stage diameter. This would increase the nozzle ratio to around 100:1, significantly better than the ratio of the 450 s versions of the RL-10, and with a higher chamber pressure.
The nozzle extension and the necessary lengthening of the interstage would add mass -- 5 tons or so, at a guess -- but even with that penalty, achieving 450 second specific impulse on the third stage could increase translunar payload by around 2 tons.
It's not particularly surprising that this approach wasn't used for Saturn/Apollo; developing and deploying a variant engine would come at a cost that might be better spent elsewhere.
- Was this possibility at least considered during the Apollo program, or during post-Apollo proposals for further Saturn development?
- Is the difference in ISP between J-2 and RL10 attributable primarily to nozzle expansion ratio, or is propellant mix ratio or another factor contributing significantly? Would a 100:1 expansion ratio J-2 be able to achieve 450 s specific impulse at 5260 kPa chamber pressure?