The nuclear test ban treaty bans testing nuclear weapons. It does not ban nuclear reactors. A nuclear rocket engine of the type proposed by NASA would be a thermal nuclear rocket:
https://en.wikipedia.org/wiki/Nuclear_thermal_rocket
This is effectively a nuclear reactor where the propellant acts as the coolant. The propellent (often liquid hydrogen) is heated to very high temperatures inside the reactor and is then allowed to escape this provides thrust as well as carrying away the reactor heat, whilst leaving the reactor core intact inside the rocket engine.
Atmospheric nuclear tests were banned in the US in 1963, but nuclear rocket engines were physically tested after that in the US under the NERVA program:
https://en.wikipedia.org/wiki/NERVA
Multiple tests of the NERVA thermal nuclear rocket were carried out at the Nevada test site in the 60’s some at more than 1 GW power and producing a specific impulse of 811 seconds (compared to chemical rockets which are limited to around 450 seconds at best).