Final Jeopardy (23 page)

Think of a billionaire selecting his outfit for a black-tie event. He can assign some tasks to his minions. One can buy socks while others track down shoes, pants, and a shirt. Those jobs, in computer lingo, run in parallel. But when it comes to getting dressed, the work becomes sequential. The man must place one leg in his pants, then the other. Maybe a few butlers could help with his socks simultaneously and hold out the arms of his shirt for him, but such opportunities are limited. This sequence, to the last snap of the cuff links, takes time.

Inside Watson, some of the sequential algorithms gobbled up a quarter of a second, half a second, even more. And they could not be shared among many machines. Watson, in all likelihood, would need the same two to five seconds by the date of the final match. At this point, the only path to greater speed was to come up with simpler commands—smarter algorithms that led Watson through fewer steps. But Ferrucci didn't expect advances of more than a few milliseconds in the coming months. Nonetheless, he found it hard to make his case to the
Jeopardy
team. From their perspective, Watson had risen from a slow-witted assortment of software into a champion-caliber player in two years. Who was to say it wouldn't keep improving?

In this jittery home stretch, it was becoming clear, the two sides shared parallel fears. While Hollywood worried that the computer would grow too smart, the IBM team focused on its vulnerabilities and fretted that it would fail. Watson's weekly blunders in the sparring sessions added to the long lists of bugs to eliminate, mauled pronunciations to remedy, potential gaffes to program around. There wasn't enough time to address them all. In the same pragmatic spirit that had marked the entire enterprise, they carried out time-benefit analyses on their list of items and focused on the ones at the top. “This is triage,” said Jennifer Chu-Carroll.

One small but vital job was to equip Watson with a profanity filter. The machine had already demonstrated, by dropping the F-bomb on its answer panel, how heedless it could be to basic norms of etiquette and decency. The simplest approach would be to prohibit it from even considering the seven forbidden words that George Carlin made famous in his comedy routines, plus a handful of others, including ethnic and racial slurs. It would be easy to draw up a set of rules—heuristics—to override the machine's statistically generated candidate answers. But what about words that included no-no's? Consider this 2006 clue in the category T Birds: “In North America this term is properly applied to only 4 species that are crested, including the tufted.” Would a list of forbidden vulgarities impede Watson from answering, “What is a titmouse?” Researchers, said David Gondek, would have to come up with “loose filters,” leaving room for such exceptions. But they were sure to miss some.

Then there was the matter of pronunciation. Watson could turn an everyday word into a profanity with just a slip of its mathematically programmed tongue. This was even more likely with foreign words. How would it fare, for example, answering this 2007 clue in the Plane Crazy category? “In 1912 this Dutch plane builder set up a plant near Berlin; later, his fighter planes were flown by the Red Baron.” This would likely be a slam-dunk for Watson, but leading it to correctly enunciate “What is Fokker?” would involve meticulous calibration of its vowel pronunciation. Surely, some would say,
Jeopardy
would not include a Fokker clue in a match involving a machine. But that would revive Ferrucci's key concern: that
Jeopardy
would be customizing the game for Watson. In the end, Watson's scientists could only fashion a profanity filter, make room for the most common exceptions, tweak potentially problematic pronunciations, and hope for the best. If the machine, despite their work, found a way to say something outrageous, it would be up to the show's producers to bleep it out.

While her colleagues steered Watson away from gaffes, Chu-Carroll was concentrating on Final Jeopardy, an area of mounting concern for Ferrucci's team. Final Jeopardy was often decisive. Throughout Watson's training, the team had studied and modeled all of the clues as a single group. They knew from the beginning that the Final Jeopardy clues were trickier—“less direct, more implicit,” in Chu-Carroll's words—but their data set of these clues was much smaller, only one sixty-first of the total. Because of this, the computer was still treating the Final Jeopardy clue like every other clue on the board, coming up with its answer in three to five seconds—and then just waiting as the thirty-second jingle went through its sixty-four notes. This was enough time for trillions of additional calculations. Wasn't there a way to take advantage of the extra seconds?

The team was not about to devise new ways to find answers. That would require major research. But Watson could take more time to analyze the answers it collected. The method, like most of Watson's cognitive work, would require exhaustive and repetitive computing. The idea was to generate from each answer a series of declarative statements, then check to see if they looked right. In the category English Poets, for example, one recent Final Jeopardy clue had read: “Translator Edward Fitzgerald wrote that her 1861 ‘death is rather a relief to me . . . no more Aurora Leighs, thank God.'” Let's say Watson came up with measurable confidence in three potential names, Alfred Lord Tennyson, Emily Dickinson, and Elizabeth Barrett Browning. It could then proceed to craft statements, putting each name in the following sentences: “_____ died in 1861,” “_______ wrote Aurora Leigh,” “_______ was an English poet.” Naturally, some of the sentences would turn out to be foolish, perhaps: “_________ found relief in death” or “________ died, thank God.” In any case, for each of dozens of sentences, Watson would race through its database looking for matches. This represented an immense amount of work. But the results could boost its confidence in the correct response—“Who is Elizabeth Barrett Browning?”—and guide it toward acing Final Jeopardy.

James Fan, meanwhile, was going over clues in which Watson failed to understand the subject. At one meeting at the Hawthorne labs, he brought up an especially puzzling one. In the category Postal Matters, it asked: “The first known air mail service took place in Paris in 1870 by this conveyance.” From its analysis, Watson could conclude that it was supposed to find a “conveyance.” That was the lexical answer type, or LAT. But what was a conveyance? In all of the ontologies it had on hand, there was no such grouping. There were groups of trees, colors, presidents, even flavors of ice cream—but no “conveyances.” And if Watson looked up the word, it would find vague references to everything from communication to the transfer of legal documents. One of its meanings involved transport, but the computer would hardly know to focus its search there.

What to do? Fan was experimenting with a new grouping of LATs. At a meeting of one algorithm team on a June afternoon, he started to explain how he could prepare Watson for what he called weird LATs.

Ferrucci didn't like the sound of it. “We don't have any way to mathematically classify ‘weird,'” he objected. “That's a word you just introduced.” Run-of-the-mill LATs, such as flowers, presidents, or diseases, provided Watson with vital intelligence, dramatically narrowing its search. But an amorphous grouping of “weird” words, he feared, would send the computer off in bizarre directions, looking at more distant relationships in the clue and bringing in masses of erroneous possibilities, or noise.

“There are ways to measure it,” Fan said. “We can look at how many instances there are of the LAT in Yago”—a huge semantic database with details on more than two million entities. “And if it isn't there, we can classify it as “weird.”

“Just based on frequency?” Ferrucci said. There were only weeks left to program Watson, and he saw this “weird” grouping as a wasteful detour. In the end, he gave Fan the go-ahead. “If something looks hare-brained and it's only going to take a couple of days, you do it.” But he worried that such last-minute fixes might help Watson on a couple of clues and disorient it on many others.
And there were still so many other problems to solve.

By the end of June, two weeks after Watson graced the cover of the
New York Times Magazine,
Harry Friedman had come to a decision. The solution was to remove the man-machine match, with all of its complications, from
Jeopardy
's programming schedule. “This is an exhibition,” he said, adding that it made the “whole process a lot more streamlined.”
Jeopardy
would follow its normal schedule. The season of matches would feature only humans. Writers would follow the standard protocols. Nothing would change. The Watson match, with its distinct rules and procedures, would exist in a world of its own. In a call to IBM, Friedman outlined the new rules of engagement. The match would take place in mid-January at IBM Research. It would feature Ken Jennings and Brad Rutter in two half-hour games. The winner as in all
Jeopardy
tournaments, would be the player with the highest combined winnings.

Friedman addressed Ferrucci's concerns about writers' bias by enlarging the pool of games. Each year the
Jeopardy
writers produced about a hundred games for the upcoming season, with taping starting in July. A few days before taping, an official from Sullivan Compliance Company, an outside firm that monitors game shows, would select thirty of those games. He would not see the clues or categories and would pick two of the games only by numbers given to them. Once the games were selected, a
Jeopardy
producer would look at the clues and categories. If any of them overlapped with those that Jennings or Rutter had previously faced, or included the types of audio and visual clues that were off-limits for Watson, the category would be removed and replaced by a similar one from another of the thirty games. If a Melville category recalled one that Jennings had faced in his streak, they might replace it with another featuring Balzac or Whitman. And for Watson's scientific demonstration, the machine would play fifty-six matches throughout the fall against Tournament of Champions qualifiers. This was the best test stock
Jeopardy
had to offer—the closest it could come to the two superstars Watson would face in January.

Jeopardy
, eager for a blockbuster, had come up with a scheme to manage the risks. After months of fretting, the game was on.

DINNER WAS OVER
at the Ferrucci household. It was a crisp evening in Yorktown Heights, a New York suburb ten miles north of the IBM labs. It was dark already, and the fireplace leapt with gas-fed flames. Ferrucci's two daughters were heading upstairs to bed. In the living room, Ferrucci and his wife, Elizabeth, recounted a deeply frustrating medical journey—one that a retrained
Jeopardy
computer (Dr. Watson) could have made much easier.

Ferrucci had been envisioning a role for computers in doctors' offices since his days as a pre-med student at Manhattan College. In graduate school, he went so far as to build a medical expert system to provide advice and answer questions about cardiac and respiratory ills. It worked well, he said, but its information was limited to what he taught it. A more valuable medical aid, he said, would scoop up information from anywhere and come up with ideas and connections that no one had even thought to consider. That was the kind of machine he himself had needed.

Early in the
Jeopardy
project, Ferrucci said, not long after the bake-off, he started to experience strange symptoms. The skin on one side of his face tingled. A couple of his fingers kept going to sleep. And then, one day, searing pain shot through his head. It lasted for about twenty seconds. Its apparent epicenter was a lower molar on the right side of his mouth. “It felt like someone was driving an ice pick in there,” he said.

When this pain returned, and then came back a third and fourth time, Ferrucci went to his dentist. Probing the tooth and placing ice on it, the dentist attempted to reproduce the same fearsome effects but failed. He could do a root canal, he said, but he had no evidence that the tooth was the problem. Ferrucci then went to a neurologist, who suggested a terrifying possibility. Perhaps, he said, Ferrucci was suffering from trigeminal neuralgia, more commonly known as the suicide disease. It was a nerve disorder so painful that it was believed to drive people to kill themselves. He recommended getting the root canal. It might do the trick, he said, and save them both from the trouble of ransacking the nervous system for answers.

Ferrucci got the root canal. It did no good. The attacks continued. He went to another neurologist, who prescribed anticonvulsion pills. When he read about the medicine's side effects, Ferrucci said, “I didn't know whether to take the pills or buy a gun.” He did neither and got an MRI. But putting on the helmet and placing his head in the cramped cylinder filled him with such anxiety that he had to go to another doctor for sedatives.

He had no idea what was wrong, but it wasn't getting any better. As the
Jeopardy
project moved along, Ferrucci continued to make presentations to academics, industry groups, and IBM brass. But he started to take along his number two, Eric Brown, as a backup. “If I don't make it through the talk,” he told Brown, “you just pick up where I leave off.”

In time, Ferrucci started to recognize a certain feeling that preceded the attacks. He sensed it one day, braced himself against a wall, lowered his head slightly, and awaited the pain. It didn't come. He moved his head the same way the next time and again he avoided the pain. He asked his neurologist about a possible link between the movements of his neck and the facial pain. He was told there was no possible connection.

Months passed. The Ferruccis were considering every option. “Someone told us we should get a special mattress,” said Elizabeth. Then a friend suggested a specialist in craniofacial pain. The visit, Ferrucci learned, was not covered by his insurance plan and would cost $600 out of pocket. He decided to spend the money. A half hour into the meeting, the doctor worked his hands to a spot just below Ferrucci's collarbone and pressed. The pain rocketed straight to the site of his deadened molar. The doctor had found the connection. A number of small contraction knots in the muscle, myofascial trigger points, created the pain, he said. The muscle was traumatized, probably due to stress. With massage, the symptoms disappeared. And Ferrucci kept them at bay by massaging his neck, chest, and shoulders with a two-dollar lacrosse ball.