How common or uncommon is that solution? Can one, for example, just plausibly assume a rocket with 4 radially placed engines (central, 5th engine disabled for certain reasons) can continue flying straight on three engines with one of the side engines switched off (without knowing much more about the rocket)?
It's very common. I'd hesitate to say you can "just plausibly assume" it, but the ability to survive an engine-out is a very useful feature and it's almost free to engineer if you're assuming a cluster of 4 or more gimbaled engines. If the engines are not too widely mounted, you only need a small amount of additional gimbaling range (beyond what's needed for nominal steering) to compensate for the lost engine.
There are a few launchers with proven outboard-engine-out capability via gimbaling.
Saturn V: Apollo 6 lost not one but two outboard engines on the second stage and was able to continue ascent to a somewhat lopsided orbit on the remaining 3. The failed outboards (#2 and #3) were adjacent rather than opposed, so thrust was unbalanced. Engine #2 began leaking fuel and losing power from about 70 seconds into the second-stage burn, but struggled on for a few more minutes before failing completely at T+413, nearly four and a half minutes into the second-stage burn. An oxidizer shutoff valve for #3 was mistakenly wired to #2's controls, so when the engine management system tried to shut off propellant flow to the failed engine, it killed #3 as well. (Further exhaustive details in Rocketdyne's failure analysis document.) If both engines had been lost early in the burn, it would have been a loss-of-vehicle situation.
Falcon 9: SpaceX's CRS-1 flight lost one corner engine (on the old 3x3 Falcon 9 1.0 layout) and continued ascent.
Saturn I: Flights SA-4 and AS-101 flew successful missions with a single engine out late in first-stage flight -- SA-4 as a test, AS-101 accidentally.
One clustered-engine launcher I'm not sure of is the Proton rocket; it's had several launch failures on the first stage, but I don't think any of them were uncomplicated single engine-outs. The 6 RD-253/RD-275 first-stage engines each gimbal only in a single plane, so there are limits to how much they could contribute to compensation for a lost engine.
Soyuz might not be able to manage loss of a booster engine. Despite the way the booster bodies are angled, the engines themselves seem to be fixed parallel to the longitudinal axis of the rocket, rather than pointing through the CG; stability is maintained with small swiveling vernier rocket nozzles rather than by gimbaling the primary nozzles, so it may not have the control authority required to compensate for a lost outboard engine. In any case, the remaining fuel in a shut-down booster would go unused (there's no cross-feed facility) which would likely fail the mission even if the rocket maintained stability.
The liquid boosters on the Long March 3B are at least pointed through the CG (as discussed here), so losing one wouldn't destabilize the rocket severely; as with Soyuz, the fuel in that booster would be wasted.
By contrast, the only launcher I know of that was designed to handle an engine-out by shutting down the opposing engine was the N-1. Its first-stage engines weren't gimbaled; steering was achieved with differential throttle and grid fins. Losing an engine thus almost demanded that the opposing engine be shut down, but with 30 first-stage engines, the loss of two wouldn't be a problem after the first few seconds of flight.
