Very little of the Saturn V's mass goes translunar -- about 45 tons of it, less than 2%. Most of the mass of the rocket is propellant, which is burned and exhausted, and returns to Earth.
Even the lunar-bound portion of the Saturn-Apollo stack doesn't reach Earth's escape velocity, though it does get close. The moon hasn't escaped Earth's gravity, after all, so the spacecraft doesn't need to in order to get there.