“But what if we go a step further?” asked Larry. “Subjects who exhibit signature behaviours online pointing to potentially violent outbursts may not provide law enforcement with sufficient evidence to justify an arrest. But that’s no reason not to provide an agent-based intervention in the online space. Once ATHENA has a sufficiently large corpus of interaction patterns, we can use it to do behavioural targeting and apply inputs weighted to divert high-risk subjects towards less damaging outcomes. Or to indirectly flag them for police attention.”
Your typical disgruntled employee is a fizzing human bomb for some time before they go postal. Their social contacts are fraying, inhibitions against violence decaying: They’re muttering to strangers in bars, reading about serial killers and fantasizing bloody revenge by night. The police will never know until they explode with murderous intent. But the spam filters monitoring their communication channels will have everything they need to diagnose the downward spiral: From their increasingly disjointed mutterings to the logs of their incoming web surfing, the pattern’s all there. And with enough data, all correlations become obvious. But what Larry was proposing . . .
“We’ve had behavioural targeting ever since the nineties: ‘If you like product X, you’ll love product Y,’ because that’s what everyone else with tastes like you bought. We can configure ATHENA to apply the same sort of recommendation nudge to behaviour to bring the subject’s outputs back towards baseline. ATHENA’s already pretty good at discriminating human-content communications from non-metacognitive signals; can we take the discrimination further, reliably, and derive objective data about internal emotional states?”
You lean back in your office chair—it squeaks angrily under your weight—and stare at the dusty display case on the opposite wall.
“Say that again,” you say.
“I’m sorry, Dr. MacDonald; it’s been a big shock to all of us here . . . can’t quite believe it. The funeral’s going to be held next Thursday morning. I’m sure everyone will understand if you can’t make it—it’s a long way to come—”
Your fingers move, eyes unseeing, to open the log of your last discussion.
ADAM@Edinburgh GMT +01:00: I didn’t adjust the preferential weightings in the naive morality table. Did you?
LARRY@Cambridge MA GMT +05:00: Not me.
VERA@Frankfurt GMT -01:00: Do we have hysteresis here? There is feedback from the second-order outcomes-triggering network.
SALLY@Edinburgh GMT +01:00: I’ve been trying to get my head around the second-order table dependencies, and I really don’t understand them.
I think there’s some redundancy, but the weighting obscures it. You need to iterate to figure out what’s going on in there.
LARRY@Cambridge MA GMT +05:00: Could be there’s feedback. ATHENA keeps reweighting its own tables to comply with the changing parameter space. That’s the problem with self-modifying code: It doesn’t sign itself.
CHEN@Cambridge GMT +01:00: the bias in tit-for-tat activation is 0.04.
Yesterday it was 0.032. I checked. There’s nothing in the commit log, so it must be internal.
ADAM@Edinburgh GMT +01:00: Maybe ATHENA is just getting annoyed at the spammers for taking all her CPU cycles.
LARRY@Cambridge MA GMT +05:00: LOLspammers. Caught between a rock and a hard AI.
He’s dead now, and it’s not fucking funny anymore. “How did it happen, do you know?” you ask aloud.
“The police are still crawling all over us, and the FBI are involved, too. They won’t say much, but rumour is, the package was misdirected. It was meant for someone in the applied proteomics group—looks like some animal-liberationist crazy sent it in and it ended up in Larry’s office. It’s all a horrible mistake—”
There is a chill in your blood and ice in your bladder as you make yourself reply carefully, lying: “I agree: Of course it’s a horrible mistake. I hope the FBI catch whoever did it quickly before they”—are duped by ATHENA into sending more packages by whatever stimulus/response tuple the weighted network has identified as most efficient in returning Larry’s communication outputs back towards baseline—“kill or hurt anyone else.”
Sally stays on the call a while longer, seeking reassurance: When you end the connection, you sit and stare at the pulsing green icon with the silhouette of an old-style rotary-dial telephone for several minutes, shaken and unsure whether you trust your own instincts.
Poor fucking Larry.
You don’t know for sure, but you don’t
need
to know for an absolute fact when inference is enough: Three days ago he was getting alarmed at the rate of creep in ATHENA’s morality tables, and now he’s dead, courtesy of a misdelivered letter bomb.
Poor fucking Anwar.
It begins to make a bit more sense, and you don’t like it one little bit. His dodgy cousin—now deceased—and his phishing sideline: He’d have been planning on hosting his phishing website on a bunch of rented zombie smartphones, wouldn’t he? Leaving exactly the kind of spoor in his communications that ATHENA would be looking for, with drastically re-weighted tit-for-tat metrics in the morality code . . .
You’re on Larry’s contact list, and Anwar’s. From Anwar to what’s-his-name, the dead cousin, is another hop. Three degrees of separation. From ATHENA’s perspective, $DEAD_COUSIN might as well be a research affiliate. Or worse: Larry—and you—might be suspected of affiliation to the botnet herders $DEAD_COUSIN was paying.
You stand up, unsteadily, and go through to Reception. “I’m going out for a walk,” you hear yourself telling Laura, as you pass her desk: “I may not be back for some time.”
Then you go downstairs, out into the bright cold daylight, to try and convince yourself that you’re jumping at shadows and the panopticon singularity does not exist.
Part 3
DOROTHY: Breakdown
Earlier:
You’re scalding yourself under the hotel shower, trying to wash the feel of his fingers off you, when you hear the telltale chirp of an incoming text from your phone.
The finger-feel is everything: You tense as you massage your abrasions, trying to brush off your own awareness of how little you meant to him—not even the joyful sharing of sex with a near stranger—but the real world is outside the curtain, buzzing on the sink side like a lonely vibrator. It’s someone on your priority list: It won’t shut up. So after another minute or so, you turn off the shower and clamber out of the tub. You towel off briefly, then when your hands are dry, you carry the phone through into the bedroom, caressing it until it calms down.
BORED.
It’s from Liz. Your throat swells: You sit down on the end of the bed and give in to the sniffles for a couple of minutes.
My life is shit.
That’s a given. For a well-adjusted bi poly femme, you’re having remarkably bad luck. Stranded up here in Edinburgh, dumped by Julian—your primary—you let Liz’s insecurity drive you into . . . into . . . nothing good. But being a victim is a state of mind, isn’t it? (
Isn’t it?
) You shiver and glance at the door, dead-bolted and with the additional security of a barbed carpet wedge you bought on eBay. He’s out there, in Room 502, two floors up and one corridor over. You can feel him—or maybe it’s just the weight of your own queasy awareness pressing down on you.
Pull yourself together.
It’s not like he’s going to break in and rape you, is it? He’s just a nasty wee shite, as they say hereabouts, a misogynistic pick-up artist who’s too cheap to use a tissue.
Keep telling yourself that, Dorothy.
There’s another muted buzz from your phone, in a cadence that tells you it’s a work message. But you really aren’t in the mood for the office on-call tap-dance: you’re disturbed, lonely, and very pissed-off—partly at yourself for not spotting the sleazebag in advance, but mostly at him for being . . . what? (You don’t blame a scorpion for stinging: It’s in his nature. Instead, you deal—with bug spray and boot-heel and extreme prejudice.) You feel like an idiot because—admit it—you wanted a bit of excitement rather than a nice hot cup of cocoa and Liz. Liz isn’t
exciting
. She’s a bit clingy, and what’s left over from her compartmentalized cop-life is boringly normal: civil partnership, not swingers’ club. So you went looking for excitement, nearly overran your safeword, and now you’re projecting all over the other. Way to behave like a grown-up . . .
The phone buzzes again. Work is calling. It’s the backside of ten o’clock, according to the hotel clock radio. Responsible grown-ups who get work calls at that time of night check to see if it’s important. The hotel comps guests a yukata, so you drop the towel and wrap the robe around yourself, then wipe your eyes and grab a hair-band before you answer: With customers all the way out to the Pacific North-west, there’s always the risk of an incoming teleconference. But when you put your specs on and glance at the log, it’s just a priority-tagged wave. URGENT CASE REVIEW REQUESTED.
Oh for fuck’s sake
—you follow the link, which leads into the agency’s human-resources back-office cloud. There’s an employee profile; they’re asking you to fill out an anonymized interpersonal ethics evaluation.
Snitcheriffic,
you think, and open it, expecting to be asked to crit one of the eager-beaver banking IT managers you were meeting with this afternoon.
Instead, it opens on a mug shot of John Christie, and a quiz that, after a second of dumb-struck confusion you recognize as the PCL-R psychopathy check-list.
Hot and cold chills mesh with nauseated recognition. You cancel out of the form frantically, racking your brain for a connection. Head office booked you into this hotel, didn’t they? What did Christie say—
here on business
? Did someone
put
him here?
But how would they know
—the front desk. Your luggage problem.
His
luggage problem. They put the frighteners on you to drive you out of your comfort zone, then banged you together with him. Emphasis on
bang
.
They? Who?
It stinks. You worked for McClusky-Williams for three years before they were taken over by Accenture, and three years since—as an independent division—and there’s
no way
anyone at head office would pull a stunt like that. It’d be a Section Four Fail for starters, with five or six other ethical violations on top, and the consequences for an ethics-compliance group of failing a moral-standards audit start with drastic and go rapidly downhill—
Reluctantly, you open up the wave and follow the link back to the snitch wizard. Yes, it’s him all right. You try to cross-reference to find his employer, but there’s nothing in the system. Digging diligently, you get nowhere except that bloody wikipedia true-crime article about his long-since-hanged namesake. There’s no job number or contract associated with this job,
it just came up in the system
. Your skin crawls as you think about what it means. You prod your way through the snitch wizard, following the script: glibness/superficial charm,
check
. Cunning/manipulative,
check
. Promiscuous sexual behaviour—
now hang on a minute
: The psych text betrays an implicit polyphobic bias—reluctantly:
check
. Your stomach clenches as you work down the list. You should have seen this coming for yourself—it was all there in front of you, wasn’t it? Christie is a poster child for narcissistic personality disorder, and you walked straight into it.
The quiz vanishes, to be replaced by another inventory questionnaire, this one more mundane: It’s an appraisal that evaluates key personality traits in an executive-founder. Private-equity outfits and VCs use it to filter their trained start-up monkeys. The target is—your heart sinks—John Christie.
“What the fuck?” you mumble to yourself, just as your phone vibrates again. It’s your private personality module. You glance at the touch screen, leaving the quiz floating open in your specs. It’s Liz again:
ARE WE STILL ON FOR SATURDAY?
You flip the phone out of work personality.
YOU HOME?
you text.
YES.
CAN I COME ROUND?
After a moment, you reluctantly add:
NEED COMPANY.
There’s nothing for a minute. Then a tag pops up, showing an address book entry and a handy route map. Your heart flip-flops. All of a sudden a cup of emotional cocoa with Ms. Clingy is looking—well, you’ll get restless eventually, but right now you’re halfway to totally creeped-out and in need of hugs and reassurance.
BE RIGHT ROUND. NEED TO TALK.
Then you go hunting for clean underwear.