The American Petroleum Institute, in its standard 521, outlines limits for exposure of personnel to heat radiation from flares. As hydrocarbons and hydrogen are commonly flared, and also commonly used as rocket fuel, the data is relevant. This publication is used throughout the oil industry worldwide (and therefore is in far wider use than anything produced by any space agency.)
Here are the limits from the 1997 edition (a bit easier to interpret for the purpose of this question than the latest edition.) The odd numbers are a result of conversion from round numbers of $\frac{BTU}{hft^2}$. For comparison, solar radiation is about 1 $\frac{kW}{m^2}$.
9.45 $\frac{kW}{m^2}$ - Exposure must be limited to a few (approx. six) seconds, sufficient for escape only. May consider tower or structure provide some degree of shielding.
6.31 $\frac{kW}{m^2}$ - Emergency actions lasting up to 1 minutes without shielding but with appropriate clothing.
4.73 $\frac{kW}{m^2}$ - Emergency actions lasting up to several minutes without shielding but with appropriate clothing.
1.58 $\frac{kW}{m^2}$ - Personnel with appropriate clothing can be continuously exposed
The latest edition reduces the times for 4.73 and 6.31 to 2-3 minutes and 30 seconds respectively, and rather unhelpfully for the point of view of this question, does not specify any time for $9.45\frac{kW}{m^2}$.
Let's take an example with a popular engine. According to Wikipedia, a SpaceX Merlin 1-C engine has a thrust of $420000 N$ and a nozzle velocity of $2600 \frac{m}{s}$ at sea level, which means a propellant consumption of $\frac{420000}{2600}=161\frac{kg}{s}$, about two thirds (by mass) of which is oxygen. The rest (say $50\frac{kg}{s}$) is kerosene. The Lower Heating Value (i.e. not considering heat recoverable by condensation of water produced in combustion) of kerosene is about $43\frac{MJ}{kg}$ so the power of a merlin 1-C is about $43 \times 50 = 2150\ MW$ or $2150000\ kW$.
Let's assume we want to at the $6.31 \frac{kW}{m^2}$ distance and assume (as the API 521 standard does) that the radiation of a combustion source is identical in all directions. To keep the calculation simple, we will assume (for now) that the emissivity of the combustion source is 1: that is, perfect radiation.
We now need to calculate the radius of a sphere such that $6.31 \frac{kW}{m^2}$ radiation will be experienced from a point source of $2150000\ kW$. Such a sphere will have an area of $\frac{2150000}{6.31} = 340729\ m^2$. As the area of a sphere is $4\times \pi\times r^2$, this works out as a distance of 165 m.
Two more things to consider: First, a Falcon 9 launch vehicle has 9 engines, not one. to factor this in, we need to multiply by $\sqrt{9}=3$ so we need to be at $165 \times 3=495 m$ distance. (Say, 500m.)
Secondly, the emissivity may be quite a bit less than 1 (values for combustion with oxygen are difficult to come by) but because of the square law it won't make much difference. Opaque smoke can make quite a difference to emissivity, but most rockets burn cleanly once they are clear of the launch pad. A low value for a smokeless flare burning heavy hydrocarbon would be 0.25 ($\frac{1}{4}$) so if this is was applicable to a rocket the distance would be halved to $250 m.$
I reckon you would survive witnessing a Falcon 9 launch at a maximum radiation of $6.31 \frac{kW}{m^2}$, though quite possibly with significant burns. It's a fairly short time before the rocket is well clear of the Earth, but it would be hot and uncomfortable (painful) with 6.31 times the solar radiation in your face. I wouldn't be surprised if you turned and ran.
Most propellants are not that toxic. Perhaps the worst exhaust fumes would be from the Space Shuttle solid rocket boosters, which produced aluminium oxide in a fine white powder form which would be very bad for your lungs. I'm pretty sure the heat radiation would still be the limiting factor though.
EDIT 1: The Soyuz launcher has five (quadruple nozzled) engines, of 813 kilonewton thrust and $2.4\frac{km}{s}$ velocity, giving a total propellant consumption of $1694\frac{kg}{s}$. That is marginally more than the $9 \times 160 = 1440\frac{kg}{s}$ used by the Falcon 9. Therefore I find the claim in the comments that the launch can be watched from $400 m$ surprising, though it does not conflict with an emissivity of $0.25$. The emissivity is something of an unknown, and the cloud of debris and steam at the launch pad would shield the observer from the heat radiation until the rocket gained some height. It's still closer than I would like to be to a launch.
EDIT 2 I am receiving comments that my thermal calculations are an overestimate. I've checked the overall energy release and that at least is correct. So let's see what may be wrong:
The spherical radiation model is an oversimplification. In fact, most of the radiation will be downwards, so this would actually increase the thermal energy felt by an observer on the ground.
I took no separate account of the energy converted to thrust. Wikipedia indicates around 60% efficiency, leaving 40% energy available for emission. I checked this with my own expansion calculation:
Chamber pressure $6.77\ MPa$ (Merlin) $5.85\ MPa$ (Soyuz): consider $60\ atm$ (approx. $6\ MPa$) for convenience.
Specific heat ratio: Both CO2 and H2O are around 1.3.
Heat not converted to thrust = $\frac{T2}{T1} = 60^{\frac{1-1.3}{1.3}} = 0.389$
This is surprisingly close to the Wikipedia efficiency value.
Given the general uncertainty of emissivity values, I do not consider a factor of 40% to be particularly significant.
- After some thought, it occurred to me that perhaps the most important difference between a flare (which as a combustion engineer in the oil industry I am very familiar with) and a rocket engine (which I am admittedly less familiar with) is the much greater turbulence with ambient air. This may lead to much greater mixing and a consequently lower emissivity.
I'm reluctant to make another guess at emissivity, but if it was as low as $\frac{1}{25}$ (that's just 4% of the heat released being converted to thermal radiation!) my estimate for the minimum non fatal distance from a Falcon 9 would be $\frac{500}{\sqrt{25}}=100\ m$ (at which distance your hearing would be severely damaged.)
It's notable that this is not much different from the radius of the cloud of dust and steam that forms at the launch pad. That debris cloud must be pretty hot (all that heat that doesn't get radiated has to go somewhere) so I think the risk of being killed by flying debris is irrelevant, as the heat would get you anyway.