I had some old optimal control software lying around. It's a really good match for this question, so I decided I would use that to attack the theoretical underpinnings of this question.
The central question I see is, does the shuttle want to roll to some non-vertical angle? Of course, by "the shuttle" I really mean the engineers who design it. And by the engineers, I really mean the underpinning mathematical reality. In short, will you burn less fuel if your trajectory is tilted somewhat? We would think the answer is "yes", but it's not easy to justify that. It's also not clear what the motivation is. If there were no atmosphere, we would prefer to launch horizontally, actually.
One way to answer this is to use numerical methods to brute-force the problem to a solution. Develop a figure-of-merit "score" function, and then vary the trajectory so that you get the best value. Numerically, you can probably accomplish this by guessing some initial trajectory and then using the calculated Jacobian and Hessian to approximately locate the nearest critical point, where the figure-of-merit is (hopefully) at a local minimum.
So to simulate, here are some of the things I take into account:
- The thrust from the engines comes out to 3gs, and this is constant
- Earth's gravity pulls it down
- Atmospheric drag follows the drag equation, and I used Falcon9 mass and diameter parameters
- The state vector gives the angles that the thrust is applied at, relative to vertical
- A simple rk4 method is used to integrate this model given a certain control
At the end of the simulation, a figure of merit function tallies its score. These are chosen arbitrarily in order to get the outcome we want. In particular, I:
- Penalize high eccentricities
- Penalize a low semi-major axis
- Penalize long burn times
The idea is that an optimizer program will find you the best combination of all these three for the lowest price. This is what I obtained in terms of the angle at which the thrust is applied. This is relative to the vertical, in the sense that 0 degrees would be applying thrust straight up. Note that thrust and velocity aren't always in the same direction. This represents the "best" way to angle the rocket to get the best performance.

This type of method does often have problems with it. A lot of the "harry" behavior there is likely due to model errors as problems with the derivative calculations and other types of artifacts. The simulation runs over 500 intervals... and there are 17 independent variables for which the second derivative determines the system. Since it's near the critical point by definition, the figure-of-merit is highly insensitive to these variables and that can cause problems.
Nevertheless, it shows what I wanted it to show - the optimal trajectory clearly follows a non-vertical path near the surface. FYI, the initial angle here is about 27 degrees relative to the vertical. I would expect that real-world scenarios would be using numbers that are somewhat close to this.
Mathematically, that is a fairly rigorous justification for the angle at which rockets fly once they're clear of the pad. It would, indeed, be more optimal to tilt the launchpad itself. Logistically, that sounds like a really bad idea. So the engineers compromise by turning to the optimal angle after the launch pad has been cleared.