My personal favorite source of Saturn V information, SP-4206 "Stages to Saturn," weighs in on the RP-1 choices.
From Chapter 7:
When the contract to build the biggest stage of the Saturn V, the S-IC first stage, was awarded to Boeing on 15 December 1961, general outlines of the first-stage booster were already fairly well delineated. The main configuration of the S-IC had already been established by MSFC, including the decision to use RP-1, as opposed to the LH2 fuel used in the upper stages. Although LH2 promised greater power, some quick figuring indicated that it would not work for the first stage booster.
Liquid hydrogen was only one half as dense as kerosene. This density ratio indicated that, for the necessary propellant, an LH2 tank design would require a far larger tank volume than required for RP-1. The size would create unacceptable penalties in tank weight and aerodynamic design. So, RP-1 became the fuel. In addition, because both the fuel and oxidant were relatively dense, engineers chose a separate, rather than integral, container configuration with a common bulkhead. The leading issue prior to the contract awards related to the number of engines the first stage would mount.
That chapter goes into a lot of detail about the design of the first stage tanks, which are enormous as-is. I think it's fair to say that even larger LH2 tanks would've compounded some of the construction problems that were had (though they could've been overcome).
Chapter 4 is more about the engines, and implies that technology readiness is a factor:
NASA's contract award to Rocketdyne in 1959, calling for an engine with a thrust of 6.7 million newtons (1.5 million pounds), was a significant jump beyond anything else in operation at the time. Executives within the space program looked on the big engine as a calculated gamble to overtake the Russians and realize American hopes for manned lunar missions. It seemed within the realm of possibility too, by using engine design concepts already proven in lower thrusts and by relying on conventional liquid oxygen and RP-1 propellants.
It's common conservatism in aerospace engineering to take incremental steps forward, so for the newly developed high-thrust F-1 engines for the first stage, they stuck with otherwise-proven hydrocarbon fuels.
I'm not immediately finding a clean source for this last claim, so it might deserve to be edited out, but I believe there's also a specific thrust advantage owing to the density of the fuel, and so for a first stage with a relatively short burn time it can be more efficient for the overall system to pay the specific impulse penalty to get the bird off the ground and out of dense atmosphere with the additional thrust, then drop the stage and switch propellants to something higher-impulse. Similar arguments apply for the strap-on solid boosters commonly used with several launch systems today.