High Thrust to Weight Ratio (TWR) rockets means get into fairly quickly and constantly experiencing G-Force, are there any downsides to high TWR rockets other than engineering? Seems like there are no problems accelerating 50 m / s^2 and get into orbit in 3 minutes.
An extremely high TWR at launch makes for a rocket that will be traveling very fast in relatively dense air. This means more energy is spent fighting aerodynamic drag, and leads to greater friction heating and aerodynamic stress on the rocket.
Note also that an initial TWR of 5:1 will increase rapidly as fuel is consumed, and implies a peak acceleration on the order of 20g or more. This is way too much for manned flights and a big issue for unmanned - most space cargos are rated for something around 6g.
Finally, there's no real advantage to getting into orbit quickly, except in the possible case of military applications. It takes months or years to prepare for a launch and it takes 92 minutes to go around once in LEO; what are you going to do with the 10 minutes saved in the middle?
In practice, the ideal TWR is usually the lowest TWR that gets the launcher safely clear of the tower in a reasonable amount of time.
Thought experiment: Consider a 75-ton launcher with a TWR off the pad of 2:1 (i.e. the engines produce 150 tons of thrust at sea level). If you were to add 25 tons of "drop tanks" - say 1 ton of plumbing and separation equipment, 2 tons of tank structure, and 22 tons of fuel - you'd raise the liftoff mass to 100 tons, and you'd have a liftoff TWR of 1.5:1. If the drop tanks feed propellant into the main tanks at the same rate the engines consume them, then at the point when you empty the tanks and drop them, you effectively have the exact same 75-ton rocket you started with (okay, with a very small penalty in plumbing mass), with full tanks, but you've added some altitude and vertical velocity - something like 7m/s for each second it took to empty the drop tanks. In practice, the mass of empty tankage is so small that real launchers don't even bother with drop tanks - they simply design for a very low initial TWR.
Falcon 9's liftoff TWR is around 1.2. Saturn V was around 1.16. Shuttle was on the high side at about 1.5.
The main disadvantage of high TWR is that engines are expensive but fuel and tankage is relatively cheap. This makes it more economical to use smaller engines and lower TWR, even though more fuel will be burned counteracting gravity the fuel is of negligible cost.
In addition, an engine capable of putting out more thrust generally needs to be larger and heavier - but that engine is not itself useful payload. Accelerating a heavy engine halfway to orbital velocity will reduce available delta-V compared with using a lighter engine which is still able to provide an acceptable TWR.
Solid Fuel, which is naturally conducive to a high TWR, also tends to be more expensive than liquid fuel. While solid fuels have their uses, it is cheaper if the bulk of the required orbital velocity comes from liquid fuels which gets back to using smaller engines at the lowest TWR that is still practical.
Higher thrust will also inflict more damage to the launchpad and surrounding infrastructure. I don't know whether that counts as a mere engineering problem, but it would certainly require a more robust launchpad or more repairs after a launch.
For manned launches, 5G is very uncomfortable, so manned rockets are often designed with a lower TWR and an ascent profile where engines are shut down to keep acceleration within limits.
This is less of an issue for unmanned rockets, but at some point G-loads will impose structural demands on the spacecraft that will make it heavier than it needs to be.
As a spacecraft uses propellant, it loses weight which means the thrust-to-weight ratio of the spacecraft continues to increase. In many cases half or more of the weight could be propellant, meaning that it can increase substantially.
This make it more difficult to do carefully controlled thrust maneuvers, such as small delta-v corrections. These are usually executed by time, and the smaller the duration, the larger the possible error, depending on the level of control and stability and repeatability of ignition.
So a potential disadvantage would be increased difficulty with small delta-v maneuvers, or the necessity of developing, testing, and calibrating additional throttling capability, or the addition of additional small thrusters that might not have been necessary with a lower TWR engine.
Because if you have a high TWR you might as well put more relatively cheap fuel in and get more deltav. Spacex does exactly that with the falcon 9, they have high TWR engines with a long fuel tank and they even super chill their fuel to get even more in. A high TWR rocket would encounter less gravity drag but the amount of delta v you save wouldn't be nearly as much as more fuel would add. There are other reasons but this is the main one.