The trouble was that humans kept winning. That was a big problem because chess researchers estimate that from any given position, even a top-ranked player needs about fifteen seconds to think through each possible move. By contrast, the early chess programs could try out thousands of moves per second. How could humans ever win? When Garry Kasparov, the world champion at the time, first played IBM's famous Deep Blue program in 1996, the computer was evaluating 100 million positions per secondâand Kasparov still won. A year later the computer had been upgraded to evaluate 200 million positions per second, and Deep Blue finally won the six-game match: two games to one, with three draws.
Yet in light of its staggering advantages, why would the computer lose or draw even a single game against any player, ever? The answer is that the human possessed something the computer didn't, which was vast knowledge of chessâhow previous masters had responded to particular positions in many different cases, and what kinds of choices generally produced what kinds of consequences. Eventually researchers from a broad array of fields realized where the secret lay. “The most important ingredient in any expert system is knowledge,” wrote three eminent scientists who work on expert computer systems (Bruce G. Buchanan, Randall Davis, and Edward A. Feigenbaum). “Programs that are rich in general inference methodsâsome of which may even have some of the power of mathematical logicâbut poor in domain-specific knowledge can behave expertly on almost no tasks.” Their conclusion: “In the knowledge resides the power.”
As it happened, other researchers were arriving at the same place by a different route, though they also were studying chess. A Dutch psychologist named Adriaan de Groot compared world-class players with good club-level players and found, surprisingly, that the world-class players didn't consider more possible moves than the less-accomplished players, nor did they search any deeper (more moves into the future), nor were their rules of thumb for choosing moves any different. In sum, their intellectual engines didn't seem to be turning any faster. So what made them better?
Part of the answer, which seems to apply in every domain, is that they had more knowledge about their field. In chess, researchers have found (using a method I'll describe a little later) that master-level players possess more chess knowledge than good club-level players by a huge margin, a factor of ten to one hundred. Just as important, top performers in a wide range of fields have better organized and consolidated their knowledge, enabling them to approach problems in fundamentally different and more useful ways. For example, accomplished physicists and beginning physics students were given two dozen physics problems and asked to sort them by type of problem. The beginners sorted the problems according to their most obvious features, such as whether they involved friction or an inclined plane. The more expert physicists sorted them by the basic principlesâsay, Newton's second lawâthat would be needed to solve them.
Other studies have replicated this finding in many other fields. Expert psychological counselors sort statements from patients according to the factors most relevant for choosing therapy, while novice counselors sort by superficial details. Commercial fishermen sort the creatures they haul out of the ocean by criteria with high practical relevance, such as behavior or commercial value; inexperienced fishermen sort the creatures by appearance. In general, the knowledge of top performers is integrated and connected to higher-level principles.
The same phenomena seem apparent in business. Many companies work hard to give their top performers the widest possible knowledge by assigning them to many jobs that are different in nature and locationâoperating jobs, staff jobs, all around the worldâand in this way the top performers have typically learned several, and sometimes all, of the most important parts of the business.
It's particularly significant that many of the best-performing companies explicitly recognize the importance of deep knowledge in their specific field, as opposed to general managerial ability. The distinction is the same as the one that computer scientists were dealing with years ago as they tried to create the General Problem Solver; America's business community followed much the same journey. The top business schools and many of the leading companies tried for decades to turn out excellent general managers, people who could land at virtually any organization and whip it into shape through the sheer power of the techniques they had learned. They didn't need to know much about the specific business, went the theory; they just needed to know the strategies for solving business problems.
But it turned out that wasn't how management worked at many of the most successful companies. When Jeff Immelt became GE's chief in 2001, he launched a study of the best-performing companies worldwideâthose that had grown much faster than the economy for many years and had produced excellent returns for shareholders. What did they have in common? One key trait the study found was that these companies valued “domain expertise” in managersâextensive knowledge of the company's field. Immelt has now specified “deep domain expertise” as a trait required for getting ahead at GE. He explained to the
Harvard Business Review:
“The most successful parts of GE are places where leaders have stayed in place a long time. Think of Brian Rowe's long tenure in aircraft engines. Four or five big decisions he madeârelying on his deep knowledge of that businessâwon us maybe as many as 50 years of industry leadership. The same point applies to GE Capital. The places where we've churned people, like reinsurance, are where you will find we've failed.”
Building and developing knowledge is one of the things that deliberate practice accomplishes. Constantly trying to extend one's abilities in a field requires amassing additional knowledge, and staying at it for years develops the critical connections that organize all that knowledge and make it useful. It must be noted, by the way, that the central importance of knowledge to great performance poses serious difficulties for the theory that great performance arises from innate talent, since no one is born with a vast fund of knowledge about anything.
The crucial role of knowledge demands that great performers develop one other key trait. After all, what good is a ton of knowledge if you can't remember it and bring it to bear at the critical moment?
Remembering More
You'll recall the description in chapter 3 of research on the memories of chess players. Expert players could look for just a few seconds at a chessboard with a real chess position, including as many as twenty-five pieces, and recall it perfectly, while novices could look at the same board and recall the places of only five or so pieces; but when the chess positions were random, experts could recall scarcely more than the novices. The conclusion was that top-ranked chess players did not possess incredible general memories but did possess an amazing ability to remember real chess positions. The question that we didn't address then but that begs for an answer is, how do they do it? How, specifically, are they able to remember so much? More generally, how can great performers in every realm recall more than would seem possible? Jack Nicklaus in his playing days could reportedly remember every shot he had hit in every tournament. Successful businesspeople often remember specific numbers from long-ago financial statements. Researchers find that excellent performers in most fields exhibit superior memory of information in their fields. What's the explanation?
Part of the answer came from the same research that produced that remarkable finding about the chess players. The experimentâpresenting a chess position for a few seconds and then asking experts and novices to recall itâlooked like a straightforward test of short-term memory. That's the type of memory in which we hold information very briefly, and if we're distracted by some other demanding task, we forget what we were trying to remember. Many decades of research have shown that average short-term memory holds only about seven items. The capacity of short-term memory doesn't seem to vary much from person to person; virtually everyone's short-term memory falls in the range of five to nine items.
As noted, the chess researchers found that the masters possessed only average short-term memories when it came to recalling randomly arranged pieces. Arguably more striking was their finding that even with real chess positions, the masters had only average short-term memories in that they recalled only five to nine “items,” just like the novices. The difference had to be in what those “items” were.
The researchers proposed what has become known as the chunk theory. Everyone in the experiment remembered more or less the same number of chunks of information. For the novices, a particular piece on a particular square was a chunk. But for the masters, who had studied real positions for years, a chunk was much larger, consisting of a whole group of pieces in a specific arrangement.
The difference is much like the difference between letters and words. Imagine that you knew all the letters of the alphabet but had no idea that they could be assembled into words. Then suppose you were shown for five seconds an arrangement of lettersâlet's say “lexicographer”âand were asked to remember the letters in the correct order. Since you would see just a bunch of letters, you'd have a hard time remembering more than the first seven or so. But in reality you recognize those letters as a word you're familiar withâand a thirteen-letter word at thatâso you can easily remember all those letters in the correct order. You wouldn't need to study them for the full five seconds; a half-second would be plenty. Though you'd have to think a bit, you could even repeat the whole string of letters backward.
When top-level chess players look at a board, they see words, not letters. Instead of seeing twenty-five pieces, they may see just five or six groups of pieces. That's why it's easy for them to remember where all the pieces are. The analogy can be carried further. You'll recall from our previous discussion of knowledge that the very best players know ten to a hundred times more than good club players. These chunks are the units of knowledge. Researchers estimate that good club players have a “vocabulary” of about 1,000 chunks, while the highest-ranked players have a vocabulary of 10,000 to 100,000.
The chunk theory is compelling and valuable, and it can be applied very widely. But as an explanation of the many remarkable memory feats of top chess players, and, by extension, of top performers in any field, it has some problems. It does fine in explaining the immediate recall of quickly presented chess positions, which are presumed to be stored in short-term memory; storing larger chunks enables expert players to overcome that type of memory's inherent limits. But short-term memoryâobviouslyâdoesn't last long and washes out if your mind turns to something else. That's why you have to write down a phone number as soon as you hear it, and if the doorbell rings in the meantime you've probably lost it.
But now think of those chess players who play ten simultaneous games blindfolded. They can't be holding all those chessboards in short-term memory because if they were, each time they turned to the next board they'd forget the one they were just thinking about. And they can't be using long-term memory because, at least as that type of memory is conventionally defined, storing and retrieving information fast enough and reliably enough to use in a chess game is not possible. So how do these expert players do it? The answer helps to explain the exceptional performance not only of top chess players but also of the best doctors making diagnoses, computer programmers writing software, architects designing buildings, executives choosing strategies, and any other excellent performer.
All these people have developed what we might call a memory skill, a special ability to get at long-term memory, with its vast capacity, in a fast, reliable way. They aren't using short-term memory or traditionally defined long-term memory. The researchers who first proposed this explanation, Anders Ericsson and Walter Kintsch, called it long-term working memory. Other researchers have called it expert working memory. To understand its critical element, remember the story of SF, the yelling runner who was able to recall extraordinarily long lists of random digits. He did it by relating the digits to numbers in forms that were meaningful to him; for example, he recalled the digits 4 1 3 1 in the form 4:13.1, a time for running the mile. He had created what's called a retrieval structure, a way of connecting the data to concepts he already possessed.
SF was trying only to recall digits. He had no larger objective, so he created a retrieval structure from concepts that just happened to be available to him and had nothing to do with his task. In the real world, the great power of long-term working memoryâthe reason it distinguishes the best performersâis that it's built on a retrieval structure connected to the very essence of the activity. Indeed, top performers' deep understanding of their field becomes the structure on which they can hang the huge quantities of information they learn about it.
To illustrate, consider first a simple research study involving two groups: devoted baseball fans and casual observers of the game. Both groups were given an engagingly written description of a half-inning of a game. Later, the devoted fans were much better able to recall the events that mattered to the game's outcomeâadvancing runners, preventing runs scored, and so on. The casual observers tended to remember colorful but irrelevant details, such as the crowd's mood and the weather. The fans' high-level knowledge of the game provided a framework on which to hang the information they had read.
That finding applies generally: Top performers understand their field at a higher level than average performers do, and thus have a superior structure for remembering information about it. The best medical diagnosticians remember more about individual patients because they use the data to make higher-level inferences for diagnoses than average performers do. The best computer programmers are much better than novices at remembering the overall structure of programs because they understand better what they're intended to do and how. Beginners at electronic engineering look at a circuit diagram and see components, while experts see major functional groups and remember them better. Rigorous research has shown all these and many more examples.