Blue is Earth, Red is Mars, Orange is Jupiter and all the whites are Earth to Mars trajectories. The odd trajectory lines in the middle are the problem I'm trying to solve.
As you can see in the image, I have a good Kepler solver (in fact I've implemented 3 while debugging this trajectory problem, to ensure that wasn't where the issue was). But in using the Gauss Problem/Method to calculate trajectories given the position of Earth at launch time, the position of Mars at arrival time, and travel duration, there are times when the solution results in a semi-major-axis with a negative value.
My main resource for the Gauss algorithm has been this site: http://www.braeunig.us/space/interpl.htm.
Reading http://www.braeunig.us/space/orbmech.htm, it seems that the semi-major-axis is negative for hyperbolas and that hyperbolas are used when the ships velocity is strong enough to escape the gravity of its primary. So perhaps my problem isn't that my Gauss solver and Kepler to cartesian are wrong, but that the trajectory I'm trying to solve requires a different type of solution?
I think it really comes down to the question What do I do when the semi-major-axis is negative? Is there a different set of equations to get the orbital mechanics (and then convert to cartesian coordinates) for hyperbolic transfers?