Survivorship Bias
Posted on April 5, 2015 — 2 Minutes Read
With advances in fighter aircraft, the sky was a major battlefield in World War II. At the height of war, the chance of a bomber aircraft crew returning home safely after a mission was only around 50%. Abraham Wald, a renowned statistician and a member of the Statistical Research Group at Columbia University, was as such assigned the task to increase aircraft survivability. To avoid being shot down during a mission, the aircrafts needed more armour. Yet adding armour all over would be a waste of previous metal that could have been commissioned for weapons and other uses, not to mention the added weight of the aircraft would render it fuel inefficient. The question as such became where on the aircraft, specifically, to add armour to that would provide extra protection against enemy fire without weight it down all the same?
Military researchers looked at all the aircrafts that returned home safely, recorded where the bullet holes were, and recommended that armour be added to the areas that were most hit, on ground that reinforcement should be added to areas that were mostly fiercely attacked. Wald noticed nonetheless that the military research only studied aircrafts that had survived their missions, for obviously those that did not never returned, and became available for study. That meant the bullet holes in the returning aircraft, counterintuitive it might seem, represented areas where a bomber could take damage and still return home. Wald as such proposed that armour to be reinforced to areas where the returning aircraft were unscathed, since those were the areas that, if hit, could cause fatal damage to the aircraft. This was one prime example of the fallacy known as survivorship bias. Like the military researchers back then, it is easy to get hung up on one side of the story. Case in point, there are countless studies these days that look at what the successful have in common. Yet never there is one on those who are not. To stay clear of survivorship bias, it is crucial to keep in mind that those who do not survive give us an equally important lesson as those who do What doesn’t work is as important as what does.