You are exactly right. The spacecraft that want to dock with the Space station have to speed up to almost the same speed and same direction as the space station. They then approach it at a very slow relative velocity.
Likewise, when an astronaut is on a space-walk, she is moving at the same speed and in the same direction as the space station.
There are two (related) physical principles to understand.
- Velocity is relative. When you talk about something's speed or velocity you always need to say what you are measuring it relative to. On the ground we usually understand that velocity is measured relative to the surface of the Earth, but when in space you need to be explicit. The docking spaceship needs to have a velocity of 7.66km/s relative to the Earth and a velocity of 1m/s relative to the space station.
- Newton's first law states that a body will remain at the same speed unless a force acts upon it. So when an astronaut steps out she will continue at the same speed. She won't start to slow down or speed up unless a force acts upon her. She certainly won't suddenly change from travelling at 7.66km/s relative to the Earth to moving at 0km/s relative to the Earth just because she is no longer physically inside the space station.