Well, a party balloon could definitely be dropped into an atmosphere, but it's tricky to find a way for it to survive the experience.
At orbital or interplanetary velocity, the balloon would burn up in the atmosphere. It doesn't matter whether it starts out inflated or not. At those speeds the balloon has a lot of kinetic energy that is going to be converted to heat, and rubber is less heat-resistant than many other materials that also burn up in the atmosphere.
If you dropped it from a rocket platform hovering at 100 km, then I am not sure if it would pick up enough speed while air resistance was negligible to burn up in the slightly thicker part of the atmosphere. After 20 km it would be going around 1400 mph. I am thinking that as the atmosphere got thicker there would be heating at those speeds and likely destruction of the balloon, but I am not sure.
You also ask what the terminal velocity of a balloon is. The terminal velocity always depends on the medium. In air at sea level, the terminal velocity is something you have observed directly; I am guessing less than one m/s. But that is not very relevant to what happens when the balloon is initially moving very fast.