We can breathe pure oxygen for unlimited time if the pressure is not too high; about 0.4 bar is okay. Breathing pure oxygen at 1 bar is possible for some hours, but a longer time may damage the lungs.
A mix of oxygen and helium is also breathable and is used for deep diving. Xenon cannot be used due to its narcotic effect on the body. Argon is less narcotic and may be used at a pressure less than about 24 bar. Krypton is also narcotic at pressures above 3.9 bar. Neon is narcotic at very high pressure above 110 bar. Radon, the heaviest noble gas should not be breathed due to its radioactivity. A mixture of several noble gases is possible if containing oxygen too.
Even a mixture of hydrogen and oxygen may be breathed. For security the mixture should not be ignitable or explosive. A mixture of 4 % oxygen and 96 % hydrogen could be used at a pressure of 5 bar or more, see hydrox.
The partial pressure of oxygen is important, it should be neither too low (less than 0.2 bar, hypoxic) nor too high (more than 0.4 bar, oxygen toxicity). This means that the percentage of oxygen might well be very different than 20%; what is important is that each lungful have neither too much nor too little oxygen, with the other gases causing no harm. A mixture with only 5% oxygen is breathable if the total pressure is 4 bar or more, but less than 8 bar.
An atmosphere with 21 % oxygen and 79 % nitrogen at a pressure of only 0.75 bar will be breathable too. That is what many aircraft passengers and personnel experience every day, also humans living in heights of about 2500 m.
Of course the content of toxic or harmful gases like chlorine, fluorine, carbon monoxide or dioxide and many others should be so low to be not harmful. Gaseous chemical compounds may be part of the mixture if they are inert for the human body and thus not toxic or harmful.
More information about breathable gas mixtures in Wikipedia.