There are two parts to an answer to this question. I am going to assume that you want to get into space with some kind of flying machine, so in particular I'm not talking about a space elevator: space elevators are a hugely cool idea but we're not very close to being able to build one.
The first thing, as other people have said, is that the main bit of getting into orbit is not getting high, but getting fast. As an example, at a height of $400\,\mathrm{km}$ (about the height the ISS orbits at), orbital speed for a circular orbit is about $7670\,\mathrm{m/s}$. This is about $22$ times the speed of sound in air.
Well, if you look at the design of aircraft which are designed to go much faster than the speed of sound you'll find several things:
- they are made of special materials as they get so hot due to forcing their way through the atmosphere;
- they burn a lot of fuel;
- they don't go anywhere near $22$ times the speed of sound – the SR-71 Blackbird could do somewhere shy of Mach 3, an it was an extremely exotic bit of machinery.
And things get a lot worse as you go faster than that. And to get into orbit you need to go about eight times as fast as that.
What this means is that, if you want to get into orbit, you have to get out of the atmosphere as fast as you can, because you really do not want to be designing a vehicle which can travel at anything like orbital velocity in the atmosphere unless it's doing that in order to slow down from orbital velocity. Indeed it's almost certainly not possible to construct such a thing with materials we have, and if it was it would burn a stupidly large amount of fuel.
This is why rockets go straight up initially, for instance: they want to get out of the atmosphere as quickly as they can so they can do most of the acceleration they need to do out of the atmosphere.
So the first conclusion is that getting to orbit means getting out of the atmosphere really quickly: once you've started accelerating to orbital velocity you need to be out of the atmosphere as soon as possible.
Well, the result of this is that, for most of the passage into orbit, you are doing it in something which is more-or-less a vacuum. In particular you can't rely on the air for lift or for reaction mass, because there is (almost) no air.
But why does this mean you have to get into orbit quickly? Why couldn't you, for instance, just drift gently up to orbital height and then zoom off sideways? Or not bother with the whole orbit thing at all and just go vertically upwards all the way to Mars or something?
Well, the answer to that is that is a thing called $\Delta v$, which is, in simple terms, the amount of speed change you need to do something. Let's consider a really simple rocket: it will go vertically upwards until it gets to where it wants to go, then it will stop. We can give a slightly hairy expression for the $\Delta v$ needed (the maths here gets simpler in a bit):
$$\Delta v = \int_{t_0}^{t_1}\left|\frac{GM}{r^2(t)} + \ddot r(t)\right|\,dt$$
Here $r(t)$ is the distance of the rocket above the centre of the Earth, as a function of time, and $t_0,t_1$ are the times it start & stops. $\ddot r(t)$ is the acceleration of the rocket, and the $GM/r^2(t)$ term is the part due to gravity.
Well, let's assume that $r(t) = ut, u > 0$: the rocket just goes upwards at a steady speed. Then we can actually do this integral:
$$
\begin{align}
\Delta v
&= \int_{t_0}^{t_1}\left|\frac{GM}{r^2(t)} + \ddot r(t)\right|\,dt\\
&= \int_{t_0}^{t_1}\frac{GM}{u^2 t^2}\,dt\\
&= \left.-\frac{GM}{u^2 t}\right|_{t_0}^{t_1}\\
&= \frac{GM}{u^2}\left[\frac{1}{t_0} - \frac{1}{t_1}\right]
\end{align}
$$
And let's say we want it to go from $h_0$ to $h_1$ above the surface, corresponding to
$$
\begin{align}
t_0 &= \frac{R + h_0}{u}\\
t_1 &= \frac{R + h_1}{u}
\end{align}
$$
This gives us the following expression for $\Delta v$:
$$\Delta v = \frac{GM}{u}\left[\frac{1}{R + h_0} - \frac{1}{R + h_1}\right]$$
Well, the detail of this is not very important, but the important thing is that there is a factor of $1/u$ in this expression: the lower $u$ is the higher the $\Delta v$ is. Let's write
$$\Delta v = \frac{\Gamma}{u}\quad\text{where }\Gamma = GM\left[\frac{1}{R + h_0} - \frac{1}{R + h_1}\right]$$
And $\Delta v$ translates into fuel in a rather horrible way. The important quantity to think about is the ratio of the initial mass, $m_o$ to the final mass, $m_f$ of the rocket: this is the ratio of how much mass you start with to how much mass you end up with. If it's a big number then the rocket you start with has to be that much bigger. This ratio can be given in terms of $\Delta v$ using the Tsiolkovsky rocket equation
$$
\begin{align}
\frac{m_o}{m_f}
&= e^{\frac{\Delta v}{v_e}} &&\text{$v_e$ is exhaust speed}\\
&= e^{\frac{\Gamma}{v_e u}}\\
\text{or}\\
m_o &= m_f e^{\frac{\Gamma}{v_e u}}
\end{align}
$$
OK, so what does this mean? What it means is that the initial mass of the rocket goes exponentially as $1/u$, which is the ascent speed. Well, speed is distance over time, so the initial mass of the rocket goes exponentially as the time taken to ascend.
This is why rockets want to get into orbit quickly: not getting to orbit quickly absolutely catastrophic in terms of fuel requirement!