of skill and luck to a particular event. If what happens is mostly the result of skill, then reversion toward the mean is scant and slow. If you're a highly skilled NBA player making free-throw shots, your shooting percentage will stand well above the average most of the time. Sometimes your performance will move back toward the average, but not by very much. If the outcome is mostly due to luck, reversion to the mean will be pronounced and quick. If you're playing roulette and win five times, you're better off leaving the table, because you can be sure you're going lose as the number of plays increases. These concepts are important and are often overlooked in business, sports, and investing, not to mention in the casino.
Take another example from sports. Tennis is largely a game of skill. Top professional men players hit in excess of six hundred shots during a best-of-five set match, providing plenty of opportunity for skill to shine through (large sample). As a consequence, the ranking of the best tennis players tends to persist from year to year. For instance, Roger Federer, one of the greatest players of all time, spent a total of 288 weeks—longer than five years—in the number-one spot. A look at the four top-rated players at the end of 2010 reveals that they were the same as at the end of 2009, with the only difference being that the top two players swapped spots. The same four players appeared in 2011. Reversion to the mean is muted because skill exerts the most powerful influence over who wins.
Baseball is another story. Even though its professional players are extremely skillful, baseball is a sport that involves a lot of luck. A pitcher can throw well but fail to get supporting runs from his teammates and thereby lose a game. A batter can put a ball into play and a slight difference in trajectory will determine whether it's a hit or an out. Over a long, 162-game season, the best teams in baseball rarely win more than 60 percent of their games, as reversion to the mean powerfully drives the outcomes back toward the average. In sharp contrast to tennis, baseball has a lot of randomness. Only the New York Yankees were one of the top four teams in 2009, 2010, and 2011 (based on wins), and they made it by a slim margin in 2010. Because there are nine defensive players on the field at any given time, and each player's performance fluctuates, one player's skill can easily be canceled out by another's mistake, driving the whole system back toward the average. So no matter how skillful the individual players, a system like this tends to look and behave much more like a game of chance than tennis does.
Naturally, for any particular individual or organization skill will change over time. The performance of a great athlete fades with age and a company's competitive advantage eventually gets whittled away. But from period to period, a sense of the ratio of skill to luck is of great value in anticipating the rate of reversion to the mean.
Interactions Vary, but the Lessons Remain
Some of the interactions featured in this book are focused on the individual, including cognitive tasks (music), physical tasks (gymnastics), or tasks in which an individual interacts with a system (the lottery). These activities tend to have a high degree of independence, which means that whatever happens next is not influenced by what happened in the past. In those cases, the skill of the players tends to dictate the results.
Still other activities have one person or entity competing against a few others. A company launching a new product amid a handful of rivals is one example. So is a team competing in a league, or even the performance of a player on a team. In these instances, what happened in the past does influence the future, a process known as path dependence.
Finally, there are cases in which one person competes with a crowd. Examples include betting on sports and investing, where an individual pits his or her skill against the collective skill of the crowd. History shows us that crowds can be wise or whimsical.
So far, I have depicted events as if they follow distributions that are known. For example, de Moivre's equation applies to events that follow a normal, or bell-shaped, distribution but doesn't apply in cases where some events are extreme outliers. The real world is messy, and there are myriad distributions that depart from the simple bell curve, as we will see. But if we approach these activities properly, the effort of untangling skill and luck will yield insights into how to assess past events and anticipate the future.
Limits of the Methods
Nassim Taleb offers a useful way to figure out where statistical tools are likely to work and where they fail. He introduces a 2×2 matrix, where the rows distinguish between activities that can have extreme variation and those that have a narrower range of possibilities.30 The narrow distributions are the ones that de Moivre's equation handles superbly. The distribution of stature is a classic example, as the ratio between the tallest and shortest human on record is only 5:1. But extreme variation is a lot more difficult to deal with. For example, the distribution of wealth has extreme outcomes. The net worth of Bill Gates, in excess of $50 billion, is more than 500,000 times more than the median net worth of all Americans.
The columns of the matrix are the payoffs, and distinguish between the simple and the complex. Binary payoffs are simple: the team wins or loses; the coin comes up heads or tails. Again, modeling these payoffs mathematically is relatively straightforward. Complex payoffs would include the casualties from a war. You may be able to predict a war, but there's no reliable way to measure its effect. Figure 1-2 summarizes the matrix.
FIGURE 1-2
Taleb's four quadrants
Source: Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable (New York: Random House, 2010), 365.
Statistical methods tend to work well in quadrants one through three, and most of what we will be dealing with falls into one of those quadrants. Dealing with quadrant four is far more difficult, and there is a natural and frequently disastrous tendency to apply naively the methods of the first three quadrants to the last. While most of our discussion will dwell on areas where statistics can be helpful, we will also discuss ways to cope with activities in the fourth quadrant.
CHAPTER 2
WHY WE'RE SO BAD AT DISTINGUISHING SKILL FROM LUCK
AS PART OF A LECTURE that he delivers to the general public, Simon Singh, a British author who writes about science and math, plays a short snippet from Led Zeppelin's famous rock song, “Stairway to Heaven.” Most of the people in the audience are familiar with the tune, and some know the lyrics well enough to sing along.
He then plays the same song backward. As you would expect, it sounds like gibberish. He follows by earnestly asking how many heard the following lyrics in the backward version:
It's my sweet Satan. The one
whose little path would make me
sad whose power is Satan.
Oh, he'll give you, give you 666.
There was a little toolshed where
he made us suffer, sad Satan.
The words are a little odd, but the satanic theme is clear. Even so, no one in the audience had heard those words the first time through. But then Singh replays the backward clip, and this time he displays the pseudo lyrics on a screen and highlights them so that everyone can follow along. And sure enough, the audience unmistakably hears the words, where before they had heard nothing. The first time through, the backward version was an incoherent mess. But once Singh told the audience what might be there, the previously unintelligible gibberish was transformed into clear speech.1
Singh's demonstration provides an important clue to why we have a hard time understanding the roles of skill and luck. Our minds have an amazing ability to create a narrative that explains the world around us, an ability that works particularly well when we already know the answer. There are a couple of essential ingredients in this ability: our love of stories and our need to connect cause and effect. The blend of those two ingredients leads us to believe that the past was inevitable and to underestimate what else might have happened.
Stories,