Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (11 page)

Why, specifically, were they targeting these folks? Vulnerability is worth gold. It always has been. Picture an itinerant quack in an old western movie. He pulls into town with his wagon full of jangling jars and bottles. When he sits down with an elderly prospective customer, he seeks out her weaknesses. She covers her mouth when she smiles, indicating that she’s sensitive about her bad teeth. She anxiously twirls her old wedding ring, which from the looks of her swollen knuckle will be stuck there till the end of her days. Arthritis. So when he pitches his products to her, he focuses on the ugliness of her teeth and her aching hands. He can promise to restore the beauty of her smile and wash away the pain from her joints. With this knowledge, he knows he’s halfway to a sale before even clearing his throat to speak.

The playbook for predatory advertisers is similar, but they carry it out at massive scale, targeting millions of people every day. The customers’ ignorance, of course, is a crucial piece of the puzzle. Many of the targeted students are immigrants who come to this country believing that private universities are more prestigious than public ones. This argument is plausible if the private universities happen to be Harvard and Princeton. But the idea that DeVry or the University of Phoenix would be preferable to any state university (much less public gems such as Berkeley, Michigan, or Virginia) is something only newcomers to the system could ever believe.

Once the ignorance is established, the key for the recruiter, just as for the snake-oil merchant, is to locate the most vulnerable people and then use their private information against them. This involves finding where they suffer the most, which is known as
the “pain point.” It might be low self-esteem, the stress of raising kids in a neighborhood of warring gangs, or perhaps a drug addiction. Many people unwittingly disclose their pain points when they look for answers on Google or, later, when they fill out college questionnaires. With that valuable nugget in hand, recruiters simply promise that an expensive education at their university will provide the solution and eliminate the pain. “
We deal with people that live in the moment and for the moment,” Vatterott’s training materials explain. “Their decision to start, stay in school or quit school is based more on emotion than logic. Pain is the greater motivator in the short term.” A
recruiting team at ITT Technical Institute went so far as to draw up an image of a dentist bearing down on a patient in agony, with the words “Find Out Where Their Pain Is.”

A potential student’s first click on a for-profit college website comes only after a vast industrial process has laid the groundwork. Corinthian, for example, had a thirty-person marketing team that
spent $120 million annually, much of it to generate and pursue 2.4 million leads, which led to sixty thousand new students and $600 million in annual revenue. These large marketing teams reach potential students through a wide range of channels, from TV ads and billboards on highways and bus stops to direct mail, search advertising on Google, and even recruiters visiting schools and knocking on doors. An analyst on the team designs the various promotions with the explicit goal of getting feedback. To optimize recruiting—and revenue—they need to know whom their messages reached and, if possible, what impact they had. Only with this data can they go on to optimize the operation.

The key for any optimization program, naturally, is to pick an objective. For diploma mills like the University of Phoenix, I think it’s safe to say, the goal is to recruit the greatest number of students who can land government loans to pay most of their tuition and
fees. With that objective in mind, the data scientists have to figure out how best to manage their various communication channels so that together they generate the most bang for each buck.

The data scientists start off with a Bayesian approach, which in statistics is pretty close to plain vanilla. The point of Bayesian analysis is to rank the variables with the most impact on the desired outcome. Search advertising, TV, billboards, and other promotions would each be measured as a function of their effectiveness per dollar. Each develops a different probability, which is expressed as a value, or a weight.

It gets complicated, though, because the various messaging campaigns all interact with each other, and much of their impact can’t be measured. For example, do bus advertisements drive up the probability that a prospect will take a phone call? It’s hard to say. It’s easier to track online messaging, and for-profits can gather vital details about each prospect—where they live and what web pages they’ve surfed.

That’s why much of the advertising money at for-profit universities goes to Google and Facebook. Each of these platforms allows advertisers to segment their target populations in meticulous detail. Publicists for a Judd Apatow movie, for example, could target males from age eighteen to twenty-eight in the fifty richest zip codes, perhaps zeroing in on those who have clicked on or “liked” links to Apatow’s hit movie
Trainwreck
, have mentioned him on Twitter, or are friends with someone who has. But for-profit colleges hunt in the opposite direction. They’re more likely to be targeting people in the poorest zip codes, with special attention to those who have clicked on an ad for payday loans or seem to be concerned with post-traumatic stress. (Combat veterans are highly recruited, in part because it’s easier to get financing for them.)

The campaign proceeds to run an endless series of competing ads against each other to see which ones bring in the most pros
pects. This method, based on so-called A/B testing, is one that direct-mail marketers have been using for decades. They send a plethora of come-ons, measure the responses, and fine-tune their campaigns. Every time you discover another credit card offer in your mailbox, you’re participating in one of these tests. By throwing out the letter unopened, you’re providing the company with a valuable piece of data: that campaign didn’t work for you. Next time they’ll try a slightly different approach. It may seem fruitless, since so many of these offers wind up in the trash. But for many direct marketers, whether they’re operating on the Internet or through the mail, a 1 percent response rate is the stuff of dreams. After all, they’re working with huge numbers. One percent of the US population is more than three million people.

Once these campaigns move online, the learning accelerates. The Internet provides advertisers with the greatest laboratory ever for consumer research and lead generation. Feedback from each promotion arrives within seconds—a lot faster than the mail. Within hours (instead of months), each campaign can zero in on the most effective messages and come closer to reaching the glittering promise of all advertising: to reach a prospect at the right time, and with precisely the best message to trigger a decision, and thus succeed in hauling in another paying customer. This fine-tuning never stops.

And increasingly, the data-crunching machines are sifting through our data on their own, searching for our habits and hopes, fears and desires. With machine learning, a fast-growing domain of artificial intelligence, the computer dives into the data, following only basic instructions. The algorithm finds patterns on its own, and then, through time, connects them with outcomes. In a sense, it learns.

Compared to the human brain, machine learning isn’t especially efficient. A child places her finger on the stove, feels pain,
and masters for the rest of her life the correlation between the hot metal and her throbbing hand. And she also picks up the word for it: burn. A machine learning program, by contrast, will often require millions or billions of data points to create its statistical models of cause and effect. But for the first time in history, those petabytes of data are now readily available, along with powerful computers to process them. And for many jobs, machine learning proves to be more flexible and nuanced than the traditional programs governed by rules.

Language scientists, for example, spent decades, from the 1960s to the early years of this century, trying to teach computers how to read. During most of this time, they programmed definitions and grammatical rules into the code. But as any foreign-language student discovers all too quickly, languages teem with exceptions. They have slang and sarcasm. The meaning of certain words changes with time and geography. The complexity of language is a programmer’s nightmare. Ultimately, coding it is hopeless.

But with the Internet, people across the earth have produced quadrillions of words about our lives and work, our shopping, and our friendships. By doing this, we have unwittingly built the greatest-ever training corpus for natural-language machines. As we turned from paper to e-mail and social networks, machines could study our words, compare them to others, and gather something about their context. The progress has been fast and dramatic. As late as 2011, Apple underwhelmed most of techdom with its natural-language “personal assistant,” Siri. The technology was conversant only in certain areas, and it made laughable mistakes. Most people I know found it near useless. But now I hear people talking to their phones all the time, asking for the weather report, sports scores, or directions. Somewhere between 2008 and 2015, give or take, the linguistic skills of algorithms advanced from pre-K to middle school, and for some applications much higher.

These advances in natural language have opened up a mother lode of possibilities for advertisers. The programs “know” what a word means, at least enough to associate it with certain behaviors and outcomes, at least some of the time. Fueled in part by this growing linguistic mastery, advertisers can probe for deeper patterns. An advertising program might start out with the usual demographic and geographic details. But over the course of weeks and months it begins to learn the patterns of the people it’s targeting and to make predictions about their next moves. It gets to know them. And if the program is predatory, it gauges their weaknesses and vulnerabilities and pursues the most efficient path to exploit them.

In addition to cutting-edge computer science, predatory advertisers often work with middlemen, who use much cruder methods to target prospects. In 2010, one effective ad featured a photo of President Obama and said: “
Obama Asks Moms to Return to School: Finish Your Degree—Financial Aid Available to Those Who Qualify.” The ad suggested that the president had signed a new bill aimed at getting mothers back in school. This was a lie. But if it spurred people to click, it served its purpose.

Behind this misleading headline, an entire dirty industry was beavering away. When a consumer clicked on the ad, according to a ProPublica investigation, she was asked a few questions, including her age and phone number, and was immediately contacted by a for-profit school. These callers didn’t give her any more information about President Obama’s new bill, because it never existed. Instead they offered to help her borrow money for enrollment.

This kind of online targeting is called “lead generation.” Its goal is to come up with lists of prospects, which can be sold—in this case, to for-profit universities. According to the ProPublica report, between 20 and 30 percent of the promotional budgets at
for-profit colleges go to lead generation. For the most promising leads,
colleges will pay as much as $150 each.

One lead generator,
Salt Lake City–based Neutron Interactive, posted fake jobs at websites like Monster.com, as well as ads promising to help people get food stamps and Medicaid coverage, according to David Halperin, a public policy researcher. Using the same optimization methods, they would roll out loads of different ads, measuring their effectiveness for each demographic.

The purpose of these ads was to lure desperate job seekers to provide their cell phone numbers. In follow-up calls, only 5 percent of the people showed interest in college courses. But those names were valuable leads.
Each one was worth as much as $85 to for-profit colleges. And they would do everything in their power to make that investment pay off. Within five minutes of signing up, according to a
US Government Accountability Office report, prospective students could expect to begin receiving calls. One target received more than 180 calls in a single month.

The for-profit colleges, of course, have their own methods for generating leads. One of their most valuable tools is the College Board website, the resource that many students use to sign up for SAT tests and research the next step in their lives.
According to Mara Tucker, a college preparedness counselor for the Urban Assembly Institute of Math and Science for Young Women, a public school in Brooklyn, the search engine on the website is engineered to direct poor students toward for-profit universities. Once a student has indicated in an online questionnaire that she’ll need financial aid, the for-profit colleges pop up at the top of her list of matching schools.

For-profit colleges also provide free services in exchange for face time with students.
Cassie Magesis, another readiness counselor at the Urban Assembly, told me that the colleges provide free workshops to guide students in writing their résumés. These ses
sions help the students. But impoverished students who provide their contact information are subsequently stalked. The for-profit colleges do not bother targeting rich students. They and their parents know too much.

Recruiting in all of its forms is the heart of the for-profit business, and it accounts for far more of their spending, in most cases, than education. A Senate report on thirty for-profit systems found that they employed one recruiter for every forty-eight students. Apollo Group, the parent company for the University of Phoenix,
spent more than a billion dollars on marketing in 2010, almost all of it focused on recruiting. That came out to $2,225 per student on marketing and only $892 per student on instruction. Compare that to
Portland Community College in Oregon, which spends $5,953 per student on instruction and about 1.2 percent of its budget, or $185 per student, on marketing.