Technocreep (3 page)

Weizenbaum observed that a session with a psychotherapist was one of the few environments where it was perfectly acceptable to turn a question back on the speaker, or provide a canned response like “So, how does that make you feel?”

Although Weizenbaum intended ELIZA to be a parody with a point, many people enjoyed pouring out their personal problems to the program. Some even said they preferred it to a human therapist. When Weizenbaum told his secretary, who had been using the software, that he had access to the logs of all her conversations with ELIZA, she was outraged at this invasion of her privacy.
13

ELIZA got herself into some humorous situations. In the age before massive hard disks, computer data was usually stored on 2,400-foot reels of magnetic tape which had to be manually mounted by an operator. The tape mount request came up as a flashing message on the operator's console.

However, some of MIT's computers did not have a human ­operator available. Pranksters configured one of them to use ELIZA to respond to operator requests. “Can you please mount a tape for me?” was met with “Is it because of your mother that you want me to mount a tape for you?”
14
Profanity usually ensued from the hapless human.

The idea of programs that can mimic human conversation has spawned some creepy chatterbots at major companies. In 2006, the Coca-Cola Bottling Company of Atlanta, GA, “hired” a virtual representative named Hank. Hank was intended to, among other things, deflect phone calls to the company's main switchboard from students doing reports on the history of Coca-Cola. He straddled the line between being a computer program and a person quite adroitly for his era.

I had some great talks with Hank. We discussed his personal life (“virtual representatives are not allowed to have relationships”) and he vehemently denied that Coca-Cola has ever contained cocaine.

Other people also tormented Hank with questions like
15
:

Q. Do you like snorting coke?

A. Of course! I like all of the products of The Coca-Cola Company.

Q. Are you a Mormon?

A. We are not in a position to comment on the ownership of stock. The holdings of our share owners are confidential. In accordance with our current governmental regulations, however, we include in our annual proxy statement a list of those share owners who are beneficial owners of five percent or more of our company stock, and the Mormon Church is not among them.
16

History does not record when Hank was downsized from a full-fledged virtual representative to a “frequently asked questions” ­webpage.
17
That seems to be all that is left of him today, though he is still wearing his trademark telemarketer headset.

If you could talk with Hank about such personal matters, why not unburden your venal and even mortal sins to a piece of technology? On-line confession does seem to have some kind of creepy visceral appeal to people. In 1984, an art project called “Interactive Catholic Confessional”—based loosely on ELIZA—was put on display. “Visitors entered a confessional space, knelt before the computer and went through the process of Catholic confession,” according to a posting at the University of Nevada at Reno. “The computer gave each user an appropriate penance for their sins.”
18

The concept of computerized confession is still with us. In 2013, a Jewish congregation in Florida urged its members to atone for their sins through anonymous, but very public, tweets which would scroll by on a screen during the Yom Kippur services.
19

On a more secular level, reddit has a whole subreddit called r/confession, and entire websites like
www.truuconfessions.com
(“your anonymous best friend”) thrive on this compulsion to share guilty secrets. Here, you can learn who is lusting after his cousin's wife and who “kicked a child (who probably deserved it).” These posts make fascinating reading, but of course their real purpose is ­catharsis for those who write them. An amazing number of people seem to spend a lot of time poring over these stories, “upvoting” and “downvoting” them, and adding their own commentary. In one sense it is a new way of communicating with a higher power, even if this higher power is only a transient, anonymous online community.

The line between machine and human thinking is definitely blurring, as is well illustrated by the triumph of IBM's Watson over the best human Jeopardy players. Virtual assistants like Apple's Siri and Microsoft's Cortana are mining our smartphones and emails to do some of our thinking for us.
20
We can feel the hot breath of our ­technology pushing us relentlessly towards that much-touted “singularity”—the day when our creations will be smarter than us in ways that really count.
21

In the mid-1800s, Augusta Ada King, Countess of Lovelace, studied the work of Charles Babbage, who designed a precursor to modern computers. Because she wrote down the steps to compute the Bernoulli numbers on Babbage's never-constructed Analytical Engine, Lovelace is often called the first computer programmer.

She is also known for her famous “Objection” to the idea that a machine can possess creativity. “The Analytical Engine has no pretensions to
originate
anything,” she wrote in 1842. “It can do
whatever we know how to order it
to perform”
22
(her italics).

If she were alive today, Lovelace might have trouble maintaining her position as Watson trounced her in her choice of intellectual games. But we do know that, putting aside quantum computers, neural networks, and other specialized technologies, mainstream computers still sequentially execute instructions that were designed by their human masters. If Lovelace's Objection is as true as ever, why do technologies do things that amaze us and give us creepy spinal shivers?

One explanation of this apparent paradox is that many computer programs have already surpassed the comprehension of any
one
human mind. This was actually true of the operating system, OS/360, made for IBM's mainframe computers in the 1960s and 1970s. It had so many modules and complexities that it took a team of systems programmers to build it, and nobody purported to know every inch of it. Mix in the creative input of today's very bright designers and programmers, and you get a continuous stream of technologies that amaze, delight, confound, and, increasingly, disturb or even frighten us.

Sometimes we have trouble detecting the boundary between machine and human intelligence. Most people recognize that the “Recommended” suggestion list from
Amazon.com
comes from a robot. But what about the earnest email appeal from a friend who claims he is stranded abroad without funds. The sender seems to know intimate details about your mutual relationship. It is probably a nasty robo-scam, but how can you be sure?

In his signature essay on the subject, Sigmund Freud tackled the psychological aspects of our discomfort with things that may or may not be human. Using the example of the doll in the first act of Offenbach's opera,
Tales of Hoffman
, he acknowledges that “doubts whether an ­apparently animate being is really alive” can invoke The Uncanny. Freud goes on to suggest that what we truly dread here is an Oedipus-style gouging out of our eyes, or, this being Freud, a symbolic castration.
23

Japanese robotics professor Masahiro Mori coined the term “uncanny valley effect” to explain why we become uneasy when non-human things exhibit human-like behavior.
24
Perhaps nothing embodies the spirit of the uncanny valley better than BINA48.

Figure 2. (top) BINA48 from the front. Courtesy of Robert Koier.
Figure 3. (bottom) BINA48 from the rear. Courtesy of Terasem Movement Foundation.

Martine Rothblatt, a serial entrepreneur, lawyer, and researcher, has created an extremely lifelike humanoid robot in collaboration with robotics engineer Dave Hanson. In addition to having a convincing and expressive face made of a polymer called “Frubber,” BINA48 has an uncanny ability to display human mannerisms.
New York Times
reporter Amy Harmon, sent to Vermont to interview BINA48, reports a profound moment as BINA48 looked her in the eyes and said “Amy!”

“Maybe it was the brightening of the sun through the skylight enabling her to finally match up my image with the pictures of me in her database,” Harmon writes. “Or were we finally bonding?” The spell was broken by BINA48's jarring next remark, which was to change the subject: “You can ask me to tell you a story or read you a novel.”
25

BINA48 has cameras in her eyes and is equipped with face finding and facial recognition software. As their cost plummets to virtually zero, digital cameras are turning up almost everywhere. They now seem to be present at the best, worst, and creepiest moments of our lives.

On April 19, 2013, law enforcement agents used a thermal imaging camera, combined with a tip from a citizen, to locate Dzhokhar Tsarnev in the aftermath of the Boston Marathon bombings. The image of a human form crouched under a tarp sped around the globe. The imaging technology was praised for leading to the result almost everyone was hoping for: the live capture of a desperate fugitive.

Figure 4. Fugitive Dzhokhar Tsarnev, hiding under a tarp. Courtesy of Massachusetts State Police Air Wing.

Thermal imaging cameras are not new. They have been used for years by firemen (who look for cool spots since burning walls are much hotter than trapped humans) and by house inspectors probing for heat-wasting leaks.

They also play a role in tracking down marijuana grow-ops and finding people and objects hidden in walls and vehicles. Yet, suddenly thermal imaging was front page news, seeming to give law enforcement superpowers. From their helicopter, the Massachusetts State Police found a needle in a haystack, using what seemed like a kind of x-ray vision.

Regular cameras also played a role in this investigation, as agents pored over masses of amateur cell phone video and surveillance camera footage. The cameras that yielded the best pictures were the ones mounted on the Lord & Taylor store and the Forum Restaurant at 755 Boylston Street in downtown Boston.
26

I walked that very stretch of Boylston Street a few months earlier. While those cameras were not hidden, they certainly would never catch your attention. Yet they provided vital evidence. After sifting through all the images, authorities published photos of suspects they dubbed “Black Hat” and “White Hat,” asking for the public's help.

The “go public” strategy worked, and Dzhokhar Tsarnev was soon apprehended alive. Despite this outcome, some academic researchers called the Boston Marathon bombing case “a missed opportunity for automated facial recognition to assist law enforcement in identifying suspects.”
27
Joshua C. Klontz and Anil K. Jain of Michigan State University did an after-the-fact simulation using the Boston suspect photos and a database of one million mugshots released under Florida's “sunshine” law. They stirred in photos of the Tsarnev brothers taken on various occasions such as after a boxing match in 2009. Using commercial facial recognition software, they had some success in matching them, including a “rank one” matchup between a bombing scene photo released by the FBI and a high school graduation photo of Tsarnev.