Super Crunchers (21 page)

The story of Epagogix's interference with the movie business can be seen as the death of art. A major Hollywood figure recently told the head of Epagogix, “You know, you absolutely revolt me.” Copaken told me, “He kept calling me Jim Jones and saying that I wanted him to drink the Kool-Aid.” We have begun to live in a world where a statistical formula tells authors what to put in their scripts. Sorry, Mr. Ivory, the hero needs a buddy if you want us to make your movie.

This truly is a scary picture of the future—where the artist is handcuffed by the gearhead. Yet this concern ignores the other shackles that are already in place. The commercialization train left the station long, long ago. Studios have been tampering with artistic vision for decades in an effort to increase movie sales.

The biggest problem isn't that studios have been interfering; rather, it's that they've been doing it badly. I'm more scared of studio execs wielding their greenlight power based on nothing more than intuition and experience than I am of a formula. Epagogix doesn't represent a shift of power from the artist to the gearhead as much as it does a shift of power from the overconfident studio apparatchiks to people who can make dramatically more reliable interventions. When Copaken recently suggested to a powerful producer that neural predictions might be more objective because they don't have to worry about bruising the egos of stars, the producer responded, “I can be just as objective as any computer.” Copaken suggested that the producer “may be subconsciously affected by his responsibility and opportunities within the industry in a way that Epagogix is not.”

There will always be legitimate and ultimately irresolvable tensions between artistic and commercial goals. However, there should be no disagreement that it's a tragedy to
mistakenly
interfere. If a studio is going to change a writer's vision in the name of profitability, it should be confident that it's right. Epagogix is moving us toward evidence-based interference.

It's also a move toward meritocracy. The Hollywood star writer system gives inordinate weight to writers who've had a hit movie in the past. It's very hard for a newcomer to even get read. Epagogix democratizes the competition. If you have a script that's off the predictive charts, you're going to have a lot better chance of seeing it on the screen whether you're a “proven” writer or not. Even some successful writers have embraced the Epagogix method. Dick Copaken told me about a well-known writer who sought out Epagogix's help because he's trying to make the shift to directing. “He figures the best way to get a job directing,” Copaken said, “is to write a mega-successful script.”

But oh, the humanity. Imagine the crushing uniformity, the critics remonstrate, that would be produced by this brave new world of film by formula. Again, this concern ignores the present pressures toward commercial conformity. Epagogix's formula didn't create the idea of a formulaic movie. It is entirely possible that an Epagogix world of cinema would exhibit more diversity than the present marketplace.

Epagogix does not use a simple cookie-cutter formula. Its neural network takes into account literally hundreds of variables, and their impact on the revenue predictions are massively interdependent. Moreover, the neural network is constantly retraining itself. Of course, so are the experiential experts at the studios. But this is precisely the horse race of figuring out the correct weights that humans are bound to lose. The studio expert is much more likely to fall back on simpler rules of thumb that lead to even less variety.

Epagogix's financial success can actually facilitate more experimentation. If it's really true that the neural network can help raise a studio's batting average from .300 to .600, studios might have more flexibility to pursue riskier or unusual projects. All the extra cash from improved predictability might give studios more wiggle room to experiment. And while Epagogix is a great leap forward over the experientialist mode of prediction, its predictions are still bound by history. The Super Cruncher studios of tomorrow will also manufacture their own new data by experimenting.

The Super Crunching of art seems perverse, but it also represents an empowerment of the consumer. Epagogix's neural network is helping studios predict what qualities of movies consumers will actually like. It thus represents a shift of power from the artist/seller to the audience/consumer. Epagogix, from this perspective, is part and parcel of the larger tendency of Super Crunching to enhance consumer quality. Quality, like beauty, is in the eye of the beholder, and Super Crunching helps to match consumers with products and services that they'll find beautiful.

Beware of Super Crunchers Bearing Gifts

Everybody loves a freebie—you know, those little gifts that sellers send their best clients. Still, we should be worried if we see a seller treating us better than its other customers. In a world of Super Crunching, sellers' promotions are far from random. When Amazon sends you a nice desk ornament out of the blue, your first reaction should now be “Yikes, I've been paying too much for my books.”

When firms Super Crunch on quality, they tend to help consumers. However, when firms Super Crunch on price, hold on to your wallet. The dark side of customer relations management is firms trying to figure out just how much money they can squeeze out of you and still keep your business. In the old days, the firm's lack of pricing sophistication protected us from a lot of these shenanigans.

Nowadays more and more firms are going to be predicting their customers' “pain points.” They are becoming more adept at figuring out how much pricing pain individual consumers are willing to endure and still come back for more. More and more grocery stores are calculating their customers' pain points. It would be a scandal if we learned that your local Piggly Wiggly was charging customers different prices for the same jar of peanut butter. However, there is nothing to stop them from setting individualized coupon amounts that they think are the minimum discount to get you to buy. At the checkout aisle, after they have just scanned in all that information about you (including swiping your loyalty card), they can print out tailored coupons with prices just for you. This new predictive art is a weird twist on the Clinton line “I feel your pain.” They feel your pain all right; but they experience it as pleasure because the high net price to you is pure profit to them.

In a world of Super Crunching, it's going to be a lot harder to rely on other consumers to keep your price in line. The fact that price-conscious buyers patronize a store is no longer an indication that it will be a good place for you, too. The nimble number cruncher will be able to size you up in a few nanoseconds and say, “For you, the price is…” This is a new kind of
caveat emptor,
where consumers are going to have to search more to make sure that the offered price is fair. Consumers are going to have to engage in a kind of number crunching of their own, creating and comparing datasets of (quality-adjusted) competitive prices. This is a daunting prospect for people like me who are commercially lethargic by nature. Yet the same digitalization revolution that has catalyzed seller crunching has also been a boon to buy-side analysis. Firms like Farecast.com, E-loan, Priceline, and Realrate.com allow customers to comparison shop more easily. In effect, they do the heavy lifting for you and help level the playing field with the price-crunching sellers. For consumers worried about the impact of Super Crunching on price, it is both the best of times and the worst of times.

Discrimination, by Other Means

The prospect of increased price discrimination is scary enough. Even more disturbing is the notion that Super Crunching can also be used to facilitate racial discrimination. Earlier I spoke about the uncontroversial successes of data-driven lending decisions. It really is true that statistical formulas beat the pants off any discretionary system of loan officers. In part, this is because statistical formulas don't have feelings. Regression equations, unlike flesh-and-blood loan officers, cannot harbor racial animus. So the seismic shift toward centralized statistical lending and insurance decisions has largely disabled hatred as a motive for minority loan or insurance denials.

However, the shift to statistical decision making has not been a civil rights panacea. The algorithmic lending and insuring policies open the possibility for race to influence centralized policies. It is highly unlikely that the algorithm would be expressly contingent on race. It's simply too likely that a race-contingent formula would become publicly known. Yet the algorithms that are formally race-neutral have at times been challenged for facilitating a type of virtual redlining. Geographic redlining was the historic practice of refusing to lend in minority neighborhoods. Virtual redlining is the analogous practice of refusing to lend to any database group that has too many minorities. The worry here is that lenders can mine a database to find characteristics that strongly correlate with race and use those characteristics as a pretext for loan denials. Members of minority groups have challenged lending denials that rely on factors like the small size of a loan or a borrower's poor credit history as pretextual characteristics that highly correlate with race.

Our civil rights statutes prohibit race-contingent lending policies. Even if a lender found that Hispanics were more likely to default than Anglos with the same credit score, the lender could not legally condition its lending decisions on race. The lender may be tempted, however, to use Super Crunching to end-run the civil rights prohibition. As long as it uses a race-neutral means, it will be very hard to establish that race was an underlying, illicit motivation for the policy.

Such virtual redlining may also take place in the insurance context. An African-American woman, Chikeitha Owens, who was denied homeowner's insurance coverage due to her poor credit, sued Nationwide Insurance. She claimed that the company's use of her credit score history effectively created a racialized category which denied coverage to applicants who were otherwise qualified.

In fact, when it comes to affirmative action, the Supreme Court invites this kind of end run. Justice Sandra Day O'Connor as the swing vote for the Court in a series of crucial affirmative action opinions said that decision makers had to try to find “race-neutral means to increase minority participation” before implementing an affirmative action policy that was expressly contingent on race. Some schools have responded to the invitation by searching for race-neutral criteria for admission that disproportionately favor minority applicants. Some California schools, for example, now favor applicants whose mothers have not graduated from college. This admissions criterion is explicitly motivated by a goal of increasing minority enrollment. Yet Justice O'Connor's standard is an open invitation to more elaborate formulas for predicting race. In a world where express race preference is prohibited, Super Crunching opens the possibility for conditioning behavior on predicted race.

Probabilistically Public

The idea that a university or insurer could predict your race is itself just another way that Super Crunching is reducing our sphere of effective privacy. Suddenly we live in a world where less and less is hidden about who we are, what we have done, and what we will do.

Part of the privacy problem isn't a problem of Super Crunching; it's the dark side of digitalization. Information is not only easier to capture now in digital form, but it is also virtually costless to copy. It's scary to live in a world where ChoicePoint and other data aggregators know so much about us. There is the legitimate fear that information will leak. In May 2006, this fear became real for more than 17.5 million military veterans when electronic records containing their Social Security numbers and birth dates were stolen from a government employee's home. The government told veterans to be “extra vigilant” when monitoring bank and credit card records, but the risk of identity theft remains. And this risk is not just limited to bureaucratic mishaps. A laptop was stolen from the home of a Fidelity Investments employee and—poof, the personal information of 196,000 HP employees was suddenly up for grabs. Or an underling at AOL hit the wrong key and poof, the personal search information of millions of users was released onto the net.

We're used to giving the phone company or websites passwords or answers to challenge questions so that they can verify that we are who we say we are when we call in. But today, new services like Corillian are providing retailers with challenge questions and answers that you have never provided, in a matter of seconds. You might walk into Macy's to apply for a credit card and while you're waiting, Macy's will ask you where your mom lived in 1972. Statistical matching algorithms let Super Crunchers connect widely disparate types of data and ferret out facts in nanoseconds that would have taken weeks of effort in the past.

Our privacy laws so far have been mainly concerned with protecting people's privacy in their homes and curtilage (the enclosed area in land surrounding your house). To Robert Frost, home was the place where, “when you have to go there, they have to take you in.” To the Constitution, however, home is quintessentially the place where you have a “reasonable expectation of privacy.” Out on the street, the law says we don't expect our actions to be private and the police are free without a warrant, say, to listen in on our conversations.

The law in the past didn't need to worry much about our walking-around privacy, because out on the street we were usually effectively anonymous. Donald Trump may have difficulty going for a walk incognito in New York, but most of us could happily walk across the length and breadth of Manhattan without being recognized.

Yet the sphere of public anonymity is shrinking. With just a name, we can google people on the fly to pull up addresses, photographs, and myriad pieces of other information. And with face-recognition software, we don't even need a name. It will soon be possible to passively identify passersby. The first face-recognition applications were operated by police—looking for people with outstanding warrants at the Super Bowl. In Massachusetts, police recently were able to use face-recognition software to catch Robert Howell, a fugitive featured on the television show
America's Most Wanted
. Law enforcement officials tracked him down after they used his mug shot from the TV show to find a match in a database of over nine million digital driver's license photos. Although Howell had managed to obtain a driver's license under a different name, facial recognition software eventually caught up with him. This Super Crunching software is also being used to catch people who fraudulently apply for multiple licenses under different names. While not perfect, the database predictions were accurate enough to flag more than 150 twins as potential fraud cases.