Never go to sea with two chronometers; take one or three.
Dual redundancy presents a big problem. Suppose you bring two chronometers (or two computers, or two inertial measurement units). What do you do when the two devices disagree? Which do you trust? Triple redundancy solves that problem. Ignoring the possibility that two of the devices go whacko at the same time and in the same way, when one device goes whacko, the other two will still be in agreement.
Sans some very smart hardware/firmware/software, triple redundancy protects against but a single failure. Dual fault tolerance requires very smart computers or requires at least quad redundancy.
Update
In this update, I will focus on fault tolerance versus fault safety, and on common mode failures.
Fault tolerance (in NASA) means that the mission succeeds despite a failure. Dual fault tolerance means that the mission succeeds despite two independent failures. Fault safety means that a failure does not result in irreparable harm, but it does not necessarily mean mission success. There's a big difference between the two. A mission that succeeds despite a failure or two is fault tolerant. Fault safety might mean sending the astronauts/cosmonauts home. While the mission would be a failure, the astronauts/cosmonauts would still be alive after the abort.
For example, as far as NASA and Roscosmos are concerned, a vehicle bringing cargo to the International Space Station would be deemed fail-safe if a failure occurs on the vehicle and the vehicle intentionally splashes itself into the Pacific as a result. A few tens of millions of dollars of cargo might be lost, but the ISS will remain unharmed. Visiting vehicles to the ISS are required to be safe against two failures, meaning no harm to the ISS. Fault tolerance? NASA and Roscosmos don't care. Their primary concern is the continued safety of that trillion dollar asset.
The wikipedia article on redundancy is very wrong. It implies that triple redundancy is the be-all and end-all of redundancy, and that triple redundancy implies dual fault tolerance. Triple redundancy works against some single failures. Dealing with two failures is a much harder problem. A number of aircraft and space vehicles are quad redundant (e.g., the Space Shuttle, the F-16, the 747, and Orbital's Cygnus). SpaceX's Dragon uses massive redundancy (with non-rad hardened flight computers) to achieve two fail safety. Dual fault tolerance is an even tougher nut to crack. Sometimes, two failures means ending the mission and sending the crew home, or splashing an unmanned payload into the Pacific.
Finally, triple redundancy accomplishes nothing if every one of those triply-redundant systems exhibits the same common error. A good number of mishaps in space have been attributed to bad flight software or to bad commands issued to the flight software. It doesn't matter if there are one hundred flight computers if each and every one of them has the same bad code or is given the same bad command. Something bad will happen. Common mode failure is the thing that scares safety engineers the most.
Carbon-copy redundancy offers zero protection against common mode failures. The Space Shuttle used quad redundancy in its primary system to address the problem of two fault tolerance (to some failures). To combat the problem of common mode failures, the Space Shuttle had a fifth Backup Flight System. The Shuttle BFS software was built by a completely different contractor than that responsible for building the primary avionics software system. The job of the BFS (which was never used) was to bring the vehicle back to Earth. While the mission would have been a failure, the astronauts would still have been alive.
Other spacecraft have taken things even further. The Japanese HTV Abort Control Unit runs a separate set of software dedicated toward keeping the Space Station safe. The ACU uses distinct attitude sensors from those used by the primary system.
Yet others argue that the idea of a distinct backup system written by a different organization is a bad idea. Per this line of reasoning, that distinct backup removes pressure on the designers of the primary system to write their software correctly and is an extra expense that would be better spent on the primary software. Moreover, the mistakes aren't so much in the code as in the requirements and design. The designers of the backup are likely to make those very same mistakes.
