SpaceX looked at the design of a booster with fresh eyes. Their concern was cost and reusability.
Different fuels, while more efficient are much more expensive to manage (two fuel systems), and develop (two different engines).
They took a more pragmatic approach of can we build an effective booster that is cheap, vs super performant.
An RL-10 cluster as the second stage of the Falcon 9 could potentially boost performance quite a bit. But thing is, performance is pretty great with LOX/RP1. But the RL-10 is super expensive compared to the Merlin engine. Thus SpaceX kept in mind cost, at what cost performance? The results do seem to speak for themselves.
The main consequence is that Falcon is great for LEO, pretty good for GTO, but not very good for interplanetary missions, compared to RL-10 based upper stages (Atlas V, Delta 4). But that is ok, the vast majority of missions are in that set of parameters.
Now SpaceX is also developing a Raptor engine running on Methane (CH4) and LOX. A lot of speculation has gone into performance of a Raptor powered second stage. But then the cost is looked at. New infrastructure at the pad (potentially 3 pads, LC-39A, LC-40, and SLC-4W) and a basically all new upper stage. With the lifespan of Falcon 9 limited by the coming monster of BFR, it is unlikely they will spend the money.
Also, SpaceX developed the Merlin engine 1A and 1B (for Falcon 1) and the Kestrel (upper stage of F1). Then the turned to the Falcon 9 and clustered the engines (and continued updating Merlin to the current 1D+++++ they are flying now. 75Klbs thrust to 190Klbs thrust earns the ++++ moniker).
So the F9 development needed engine improvements but not a new engine, new fuel.
As you look at the move from F9 1.0 to 1.1 builds with the move from Merlin 1C to 1D hindsight suggests this was a good call.
Also, if you are going to LH2/LOX you want it in your upper stage, not your lower stage. Lower stages do better with higher thrust and increased density.
Get out of the atmosphere and turn, then let the second/third stage do the rest. Performance is for the upper stage.
Some interesting related questions: