In the early days nobody cared about upper stages, so they occasionally collided with other objects or spontaneously exploded when a pressurized tank failed. That's why they represent a significant source of space debris. See also Orbital Debris from Upper Stage Breakup on Google Books.
Since the 1990s there are standard practices in place that require passivation of unused spacecrafts remaining in orbit. That means propellant and pressurized tanks should be depleted as soon as the deployed payload is out of range.
Depending on the orbit (LEO, GTO, GEO, etc.) there are different options to dispose an upper stage. The U.S. Government Orbital Debris Mitigation Standard Practices lists them as follows:
- LEO or below: Atmospheric reentry, i.e. maneuver the stage to an orbit that will cause atmospheric drag and it will burn up eventually.
- Between LEO and MEO: Maneuver the stage to a graveyard orbit between 2,000km and 19,700km.
- Between MEO and GEO: Maneuver the stage to a graveyard orbit between 20,700km and 35,300km.
- Above GEO: Maneuver the stage to a graveyard orbit above 36,100km
The goal is primarily to avoid leaving the stage in an orbit that interferes with any operational orbit in LEO, MEO and GEO. If you cannot achieve reentry, you ideally want to park the stage in an orbit as high as possible to reduce its velocity and thus to minimize energy and debris released in case of a collision.
This also applies to other spacecraft, such as satellites at their end-of-life.
The standard practices further mention "direct retrieval", basically to catch the object and remove it from orbit. Though I don't know of any instance using this method.
Another option is to leave the upper stage attached to the payload, such that it can provide power and thrust if needed.