As Russell Borogove says, if you just go straight up you fall back. Thus I'm looking at this from a standpoint of going straight up and then burning horizontal.
I don't know what the numbers look like in the real world but I have played with trajectories almost like this in Kerbal Space Program when hauling drag monsters. (The game has no good answer for lifting large rovers, thus some very ugly rockets go up at times.) Going nearly vertically until I'm out of the atmosphere costs about 10% more delta-v than a proper flight profile. Note that this is with a thrust to weight ratio well above what is normally used for real world space launches and thus understates the penalty.
Alternately, you're looking at an interplanetary trajectory. That way you will not fall back, it's simply a matter of efficiency. My memory of the one time I tried it was that the penalty was over 20%--and that with an even higher thrust to weight ratio as I was trying to approximate a vertical cannon launch + minimal fuel to go into orbit. (Go out to the edge of the sphere of influence, a small burn to put my periapsis in the upper atmosphere, aerobrake until my apoapsis was in low orbit, the a final burn to circularize.) Note that this is even more sensitive to the thrust to weight ratio as you're spending even longer without horizontal velocity.