Of course, you save a lot of material, but I have heard some Russian rocket people say that that the main cost in rocket comes from the manufacture and testing. And for reusable rockets, you need to make better pumps, and better structures, and do major NDTesting on all components before re-launches along with decreasing the overall payload capacity by a third as in disposable rockets. Of course, Musk's point seems obvious, that a disposable aeroplane would cost a lot more. But are the economic advantages very clear?
At first glance, the answer seems to be simple and trivial. It makes intuitive sense that building a rocket from scratch for every launch and then dumping it into the ocean is more expensive than reusing that rocket. However, the answer is not that simple.
You may have read this article and other like it, from 2014, talking about the tradeoffs in reusability. The key factors are number of flights, payload reduction due to recovery hardware, and the cost of refurbishment.
Cost of Refurbishment
This is an important factor in any reusable system. It costs almost nothing to ride a bicycle a second time after riding it once. For airplanes, many flights can be done for just the cost of fuel, with minor maintenance during routine operations, and major depot maintenance after a large number of flights. But for instance, a Formula 1 car or drag racer will most likely go through a full rebuild after each race. It has not been confirmed whether rockets can be operated with the frequency and cost of refurbishment of something like planes or something closer to high-performance vehicles. If the cost to refly can stay a small fraction of the cost to build a new vehicle, reuse makes sense. If it is an appreciable chunk of the cost of a new vehicle, the economics don't work out that well.
Recovery Payload Reduction
Adding recovery-specific hardware, and reserving fuel for recovery operations reduces the rocket's total payload to orbit, sometimes significantly. This reflects poorly on the \$/kg of the rocket, which at first glance, makes reusable rockets less appealing. However, rockets are not plotted functions on a table, but actual quantized vehicles with specific capabilities. Many rockets fly with excess payload capacity, and a few vehicles have the ability to add/remove boosters for additional cost to better fit the vehicle's capability to the payload. However, that 'excess' is there for most missions. A reusable rocket can therefore use that excess to attempt downrange landings or a return to launch site. In practical situations, the $/kg impact is much less than first observed.
Flight rate is critical to the economics of reusability, since the previously mentioned factors affect how many flights a reusable vehicle must make before becoming economical. If recovery hardware is added and the cost of refurbishment is high, but the vehicle can only fly twice, then it will be less efficient than an expendable rocket. Several studies have been done on the necessary flight rates, and it usually points to 10 flights being the break even point. Flights beyond that make the reusable system even cheaper. However, those models make assumptions for refurbishment costs and tend to overestimate the impact of recovery hardware, as previously mentioned.
SpaceX promises its Falcon 9 Block 5 can fly 10 times with minimal refurbishment, with a more in-depth depot maintenance after 10, with an airframe life of 100 flights. That is worse than a plane, but still economically viable.
There have been several studies done on the short-term economics of reusability. This study by Ark contains several estimates on affordability. While they concluded that a 75% reusable rocket was cheaper per launch than an expendable rocket, their results pointed to a higher \$/kg to orbit.
The conclusion that can be made from all of this is that the economics of reusability are complicated, and more real data is needed before extrapolating to a general case.