Let's say you are standing on the Moon, with the Earth directly above your head (the moon being tidally locked with the earth. At this point you and the Moon are sharing an orbit, both experiencing the same orbital elements.
You climb into a cannon that has just enough force for you to reach escape velocity (or, maybe more appropriately, leave the Moon's sphere of influence). Pull the trigger, and force is applied orthogonal to your velocity vector.
How does your new orbit differ from the Moon's? What orbital elements changed in this process? Would your orbital period relative to the moon decrease?
These answers are all really great, but I made things too complicated by involving the moon. Instead, more simply, how do orbital elements change when force is applied orthogonal to the velocity vector of an orbiting mass? Think about a satellite in GSO.