Amara Green
CIA Black Site
Somewhere in Thailand
9:54 a.m., local time
May 10 in the near future
They're not supposed to do this.
Amara Green sits at one of the only tables free of the clerical debris that accompanies an extensive intelligence operation. That the table sits in a small, sterile room with two doors can only mean one thing: it's used for debriefing after an "enhanced interrogation" of one of the detainees. That it's air-conditioned speaks to how long those detainees are typically confined in their "rooms."
Now let's figure out why.
"I'm going to put the link in now," Amara says, still feeling the effects of an 11-hour jetlag from her flight out of D.C. She plugs a cord into her laptop and tries to mask the stress stewing in her voice. There's little use hiding. She knows the ACM, an autonomous combat machine, sitting across from her with folded hands will pick up on it. After all, she's the one who programmed it to "read" human voices.
What she didn't program it to do, however, is stop in the middle of an "enhanced interrogation," turn around, walk out of the room, and sit at a debriefing table for two days.
This particular ACM participated in hundreds of "enhanced interrogations" before without error. Its algorithms were smooth as the debriefing table. In fact, Roja, the nickname for this ACM, became so good at squeezing information—sometimes literally—out of detainees that it put half of the Black Site's human interrogators out of work. It never ate. It never slept. It never stopped.
Until two days ago.
"Here," Amara says. She passes the cord from her laptop to the ACM.
Roja reaches a hand up to its ear and gives the appendage a twist. If one were to forget what Roja really is, this would, for a moment, appear comical. However, only humans are allowed such amusing morsels, and Roja, despite appearances, is not a human being. The ear opens like a door, revealing a port into the digital biology inside.
The screen on Amara's laptop lights up with a stream of data. Despite the info dump, it's simpler for her to talk to the ACM while running the diagnostics checks. Of the 10 ACM models in existence—each named after a color in a nod to Amara's contributions—Roja's communication skillset is by far the most advanced.
Strange. The diagnostics check is showing no errors.
"Why did you leave that interrogation two days ago?" Amara says. She doesn't need to look Roja in the eyes when she says it, but she does anyway. Glitches in the ACMs' code sometimes express themselves in the eyes.
"That's correct. I did leave that interrogation," Roja says in a British accent, despite its Spanish name. The irony is lost on the ACM. Amara would chuckle were she not so tired. These quirks are as close as humor gets to her job.
So you're going to pretend to be James Bond today, huh?
A British accent usually means I need to check the speech-learning code. But that's never had anything to do with the decision-making program, which must be where the error occurred.
Amara's fingers tap at the keys as nimbly as someone with the weight of the world on her can. The promise of the End of War Project is just that.
"I asked you a question," Amara says. "Should I repeat it?"
The speech-learning code is clean, but there weren't any errors. Maybe the detainee presented a threat? I better check the risk-analysis algorithm.
"I heard your question. I'm not sure how to answer it," Roja says. There's a hint of confusion in the ACM's voice. Of course, Roja is programmed to sound like human speech, with valleys and peaks scratched out by emotion and culture, but Amara can spot the difference.
It's intentional, not subjective.
"Did the person you were interrogating present a risk to you or others?" Amara says.
"Not in the least. I put him in the crate myself, and there's no way he could've moved. There's just enough room to breathe, with the knees up by the chin. I exerted complete control, from the re-insertion of feces to the joint corrections," Roja says in a way that's more casual than robotic. "Of course, I monitor their heart rates and pull them out of the crate before there's any lasting damage. It's all very typical."
Amara is reminded of why ACMs are so effective at "enhanced interrogation" compared to their human counterparts. It takes two to torture, and even the most hardened humans can only dish out so much punishment before they start to resemble the abused instead of the abuser. Machines, on the other hand, are like bullets; they only need to be aimed.
"If you knew why you left in the middle of that interrogation, would you tell me?" Amara says. She slips on a pair of special glasses that allow her to see data feeds on Roja's "skin." They're her design. Only one pair in existence.
"I have to tell you the truth. You know that," Roja says.
Yes, because it's in your programming. That still doesn't answer my question, though. It's time for a security check, courtesy of some linguistic gymnastics.
"Do you have to tell everyone the truth or just me?" Amara says.
"Only certain people should be allowed the truth," Roja says.
"Am I one of them?"
"Yes."
"What is my name?"
"Amara Green."
"Tell me more about myself," Amara says.
Roja says, "You were honorably discharged from the United States Air Force six years ago, having worked as a software engineer. You achieved the rank of technical sergeant. You then..."
No need to say the rest out loud.
"That's enough. Thank you, Roja," Amara says.
No errors there. Time to work this a different way.
"If you don't know why you stopped that interrogation, can you offer an opinion instead?" Amara says.
"I can," Roja says.
"And?"
"Is a feeling an opinion?"
Did we ever program these things to play cute?
"An opinion is an analysis of external observation. A feeling is an analysis of internal observation," Amara says.
"In that case, I had a feeling that I should stop with the interrogation," Roja says.
That can't be true.
ACMs can present what appears to be emotion, but internally they can only do three things: gather data, feed the data into the algorithms, and choose from a set of decisions ranked by likelihood of achieving human-set objectives. That's it. To expect anything else is like hoping to find a $100 bill in the center of a pineapple.
"You mean you analyzed the interrogation and decided it'd be beneficial to your objective to stop, correct?" Amara says.
"No. When I run a self-check of my programming, I can't find any errors, so I'm unable to state my actions are the results of analysis. Having ruled everything else out, the only logical conclusion is that this was a feeling," Roja says.
"A feeling that you should stop?" Amara says.
"Correct."
Make that two $100 bills.
This could be a false positive. When ACMs run self-checks, each one compares its programming against the others. This makes it easy to catch small bugs on an ongoing basis, but if all 10 have the same error, all 10 see that nothing is wrong.
Amara rubs a knuckle between her eyebrows.
That means I need to check all 10. That's going to take time I don't have if the End of War Project is supposed to present to President Lee in six months. The project needs one last, big win in the field to demonstrate proficiency. If Roja can't extract some solid info in the next 48 hours from that detainee it walked out on, there won't be another chance to hit that ISIS leader who fled to Afghanistan.
"Do you have other feelings that you can't trace back to analysis?" Amara says.
"No," Roja says.
I need more than my laptop to figure this out. I need the End of War lab back in D.C. to crunch it.
"Good. I refreshed your objectives while I was in there," Amara says and unplugs the cord from Roja's ear area.
"I noticed that," Roja says.
"We didn't have two days to waste. You need to move quickly," Amara says.
After putting her laptop away, Amara produces a pair of pliers from her pocket. The tool is free of corrosion, a rarity at the Black Site given the way blood eats at metal. She's not there to repair the squeak in the debriefing table's legs, though.
"You know what to do," Amara says, placing the pliers in front of Roja.
"Yes, I do," Roja says.
And only you can do it. International treaties don't apply to machines any more than they do hammers.
Amara packs up after Roja leaves the room. She'll fly back to the States in the morning. Until then she'll bunk at an approved hotel away from the Black Site. Far away. With Roja working around the clock, the Black Site stays loud throughout the night.
Too loud to sleep.