#### Selection bias (2004-08-24)

This page has moved to my new website.

The CarTalk radio show has
an interesting puzzle every week and often these puzzles involve mathematics.
These puzzles can sometimes help you understand complex mathematical concepts
that are important in Statistics. In the summer, they re-use puzzlers from
earlier years, and just last week, they re-used one of my favorites. A
"nameless mathematician" during World War II was asked to help with a
military problem. A lot of bombers were not returning from their missions, so
the Royal Air Force wanted to put armor on the bombers. But where to put it?
They couldn't put it everywhere because it would be too heavy. So this
mathematician looked at the planes that returned and noted where they had
holes from enemy fire. These holes were distributed more or less randomly
throughout the plane except for two regions where there were nothing. His
recommendation was to place the armor only in those two areas where no enemy
fire was found. This seems counterintuitive, which is why it makes such a
good puzzle.

This mathematician hypothesized that any plane hit in those regions did not
survive to return. The other areas could be hit and the plane could still
limp back to safety. This is an example of selection bias. The bombers in the
study were not a random sample of all bombers, they were a sample of bombers
that returned safely.

This case was actually discussed in a delightful article in Chance magazine
(see below). That article gives the name of the mathematician (Abraham Wald)
and a few more details on the logic behind this choice. It has several other
amusing examples of selection bias, such as the average age of death among
various professions. The profession that had the worst result, with an
average age at death of only 20.7 years was "student".

An excellent book by Freedman, Pisani, and Purves also has a delightful
story about selection bias.

The original Car Talk puzzler is on the web at
cartalk.com/content/puzzler/transcripts/199838/index.html.

You can read the original answer on the web at
cartalk.com/content/puzzler/transcripts/199839/answer.html.

**A Selection of Selection Anomalies.** Wainer H, Pamer S, Bradlow ET.
Chance 1998: 11(2); 3-7.

**Statistics Third Edition.** Freedman D, Pisani R, Purves R (1998) New
York: W.W. Norton & Company.
[BookFinder4U
Link]