Luck Run Amok—Part 11

Regression Toward the Mean

A subtle issue—and one that is very important for explaining income inequality—is that even when there seems to be an objective measure of a person’s value to a company, it is often a very noisy measure.

To make the argument concrete, consider a student taking a history test. The student’s ability is her mastery of the material, measured by what her (hypothetical) average score would be if she were tested a great many times. Her performance is her score on a single test, which might be above or below her ability. Although she might know the correct answers to 80 percent of all possible test questions, she won’t get 80% correct on every single test. Sometimes her score will be above 80, sometimes below. Her performance will fluctuate around her ability. When her performance is above her ability, we can say she had good luck. When her performance is below her ability, we can say she had bad luck.

The crucial insight is that a person who gets a high score relative to others taking the test probably had good luck. A person who gets a 90 on a test with an average score of 50 is more likely to be someone with ability below 90 who was lucky than to be someone with ability above 90 who was unlucky, simply because there are so many more people with ability below 90 than people with ability above 90.

If the person who gets a test score of 90 does, in fact, have an ability below 90, she can be expected to get a lower score on a subsequent test. At the other end of the performance spectrum, someone who gets a relatively low score probably: (1) had bad luck; (2) has somewhat higher ability; and (3) will do somewhat better on the next test. This statistical phenomenon is known as regression toward the mean.