Launch sites far from the equator have to cancel out less of the Earth's rotation to get to a polar orbit, the Rocket Lab Launch Complex 1 at Mahia Peninsula is at 39 degrees south giving it an extra ~361 m/s, compared to ~465 m/s at the equator. This is ~104 m/s less velocity to cancel out compared to a launch site like the EAS launch site in Kourou, French Guiana. The fact that there is no one living north and south of the launch site means that they could launch both north and south to reach a polar orbit. The empty ocean to the east also means that they can target a large range of orbits without flying over inhabited land.
There are also convenient islands further north in the Pacific to place ground stations on to control the launcher on ascent and place weather stations on etc (mentioned in passing in this article on Electrons first test flight). I was not able to find a reference to which islands Rocket Labs used but island nations like Fiji and Samoa are quite close to a likely flight path.
New Zealand being a developed nation with good infrastructure and no imminent territorial conflicts is also a plus. This fact makes it easier to accommodate the people working on the project as well as easing the movement of materiel needed for the operation of the launch range.