According to page 47 of the Explanatory Supplement to the Astronomical Almanac, TT – TAI was exactly 32.184 seconds, by definition, at the instant 00:00:00 TAI on January 1, 1977. Since then, there have been deviations from that exact value due to hardware limitations in the atomic clocks. The supplement says that between 1977 and 1990 (when it was being written), "the deviation probably remained within the approximate limits of ±10 microseconds." Clock quality has improved significantly since then, so the deviation is much smaller now; a footnote in the IAU Standards Of Fundamental Astronomy, "Time Scale and Calendar Tools" documentation, says "GPS time remains forever 19s behind TAI, to submicrosecond accuracy."
The US Naval Observatory says GPS time "during the last several years has been within a few hundred nanoseconds" of UTC(USNO), while UTC(USNO) "has been kept within 26 nanoseconds of UTC[BIPM] during the past year". The GPS performance standard for time transfer accuracy is less than 40 nanoseconds 95% of the time, but the International Earth Rotation Service's Tech Note 36 says "since the eccentricity $e$ for GPS orbits can reach up to 0.02, consequently the amplitude of [the additional relativistic correction needed beyond the one already built into the onboard clocks] can reach up to 46 ns." (page 154).
This shows that if you really need nanosecond timing precision, you must define what you want more exactly, because the uncertainty in comparing each of these time scales with itself is bigger than that! And that's before you start trying to consider ionospheric propagation delay, which can cause hundreds of nanoseconds of error at GPS frequencies, and other similar effects. I strongly recommend learning to use software like SOFA or SPICE, rather than try writing it yourself, unless you do it just as a learning exercise to discover how hard it is to get all these details right. :)