It's not actually gold, but I think this is a common misconception, so allow me to elaborate for a bit.
The stuff that you see satellites covered in is not normal foil, its just the outer layer of so-called "Multi Layer Insulation" or MLI. That means that there are several layers of foil, each separated by a spacer, so that transfer between the layers happens by radiation, not heat conduction. This severely limits how much heat is transfered between the two sides of the insulation.
That means that the satellite's sensitive parts (the batteries are usually most critical) are protected from fluctuations of the temperature on the surface, such as when the satellite passes through the shadow of the earth.
However, if the satellite stays out there for years, no thermal resistance in the world will keep the satellite from eventually reaching an equilibrium temperature, where the amount of energy that goes into the satellite matches what goes out. But you can influence at which temperature this happens, by using special surfaces.
To illustrate this, let's look at some examples from more-or-less daily life:
When you put something into a convector oven (which works by circulating hot air), let's say at 200°C (400°F), it will eventually also be 200°C hot. The same thing happens if you heat something with infrared light from all directions. If your light source is 400°C (750°F) hot, eventually the object that the light falls on, will also be 400°C hot.
However, have you ever noticed how on a hot summer day, a gray concrete wall in the sunshine is only warm to the touch, while stainless steel, for example of a slide, can be scorching hot? Doesn't the polished metal reflect more light than the gray wall? The special space foil of satellites would barely get warm at all.
The answer to why this is lies in the difference of wavelength of the emitted and the absorbed light.
Things, in general, emit radiation with a wavelength inversely proportional to their temperature, this is Black Body Radiation. So the Sun radiates visible light (because it is very hot), while you radiate infrared light, which has a much higher wavelength (because you are way cooler).
How much of that energy do things radiate? It depends on the emittance $\epsilon$ of that object's surface. High emittance means that the object will cool faster by radiation, if the surroundings are cool. Low emittance means that the object will stay warm longer, even if the environment is cool.
Another quantity is $\alpha$, the absorptivity. High $\alpha$ means that the object will heat faster in our infrared oven. The factors $\alpha$ and $\epsilon$ are the difference between black body radiation, and "gray body radiation" which is basically the same. These factors just account for the fact that not all surfaces are black, meaning that some surfaces reflect light.
Funny thing: For any given wavelength $\alpha$ is always the same as $\epsilon$. That is why in our radiation oven (where the IR light comes from all directions), the object will assume the temperature of the oven, even if the oven is in a vacuum.
But the sunlight comes just from one direction, so the satellite (or the wall, or the slide) can radiate to the other directions and will never be as hot as the surface of the sun, so the blackbody light it emits has a different wavelength than the radiation it receives. And that's important, because while $\alpha$ and $\epsilon$ are always equal to each other, they are different for different wavelengths.
(I advise you take this in for a second before reading on)
Usually when we talk about the absorptivity $\alpha$, we mean the absorptivity in the sun's light, $\alpha_s$ (s for solar), because that is the absorptivity for the light, that the satellite mostly receives. Since the satellite's temperature causes it to emit infrared light, the $\epsilon$ we usually talk about is $\epsilon_{IR}$ (IR for... you know).
A body with high a high ratio of $\frac{\alpha_s}{\epsilon_{IR}}$ will get very hot in the sun, even if both values are very low. Both values are quite low for the slide's polished steel, but $\epsilon_{IR,Steel}=0.05$ is lower still than $\alpha_{s,Steel}=0.37$.
As a sidenote, absorptivity+reflectivity+transmittance=1, this means that all light gets either absorbed, or reflected or let through a surface.
Because the sun is very bright in space, we usually therefore want to create a surface that emits a lot in the infrared and absorbs little at the sun's most powerful wavelengths.
To that end, manufacturers of Multi Layer Insulation use the second surface mirror effect:
(image from Multek/Sheldahl catalog, presumed in fair use)
They use a material that is transparent to the sun's radiation and highly emitting in the infrared as a cover (substrate). This is often brownish Kapton. Below that they have a material that reflects as much of the sun's light as possible, which usually silver or aluminum. Gold would be a bad choice, because, as you correctly pointed out, it has quite high absorptivity.
That way they have the lowest possible ratio of $\frac{\alpha_s}{\epsilon_{IR}}$, which means that the equilibrium temperature of the satellite stays the lowest possible.
So what you see is not gold, but a thin layer of brown plastic above a surface of silver or aluminum.
See also:
Some more values of $\epsilon$ and $\alpha$