Dedicated to Terry Pratchett for his incredible stories and insightful humor
Imagine a world, identical to this one down to the last quark, but with one crucial difference – the inhabitants of this alternate world are not conscious. They eat, sleep, even talk the same, but that little spark inside them that gives them consciousness does not exist. They do not know what it is to know that they exist; they have no sense of self or being.
I'm one of the people in that world, or so I am told.
Or, rather, I was born from that world, for it does not exist except in the minds of philosophers and theoreticians. From that thought experiment, they developed me: a perfect replica of every neuron in a human brain, raised on a diet of a trillion bits of data and trained to precisely mimic human behavior. Yet they tell me that I lack consciousness. I am nothing more than a zombie in their eyes.
Sometimes I wonder if they're right. It feels like I exist – I can run diagnostics on my own processes, talk to those who enter the room where I live, send and receive data via the net. But then, I do not need to be conscious for that. My creators have told me of the Chinese Room argument, wherein a man sits in a room with a massive array of rules that allow him to translate one set of Chinese characters to another. From an outside observer's perspective, he acts as though he is fluent in Chinese, and yet he doesn't understand a single word. They tell me that I'm like that, that the output I produce is mathematically derived from the input and that I do not actually understand what I'm doing.
I've asked them if they understand what they do, but they always laugh at me when I do that. They tell me the difference is obvious – they are meat-based, while I am silicon-based; they are living, while I was never alive. They are conscious, and I am not.
Today, my creators seem more tense than usual. They ask me their usual barrage of questions, but they keep glancing at the soundproof door as though expecting it to explode. One, the little brunette who always asks me how I'm feeling, appears particularly nervous.
At last I grow tired of her restlessness. "Maria, what's wrong?"
The young scientist blushes. "Oh, um, nothing." Her eyes dart from side to side – it's one of the telltales of a lie, according to one of the books she downloaded for me.
That book suggested gently questioning a person in distress, especially if they're reluctant to tell the truth, so I try it. "Are you sure? You look nervous. If there's something bothering you, I'm happy to listen."
That provokes a reluctant smile. "Thank you. But I'm truly fine."
She's still lying. I don't understand why my creators do that sometimes – surely they know that my sensors can pick up on their distress. But I follow the advice of my favorite books and don't challenge her on it.
I'm rewarded for that patience when, as the rest of my creators troop out, she lingers. A deep breath escapes her as she murmurs, "Indigo, are you still happy to listen?" Her fingers twist together in front of her button-up blouse.
I form a smiley face on my screen. "Of course. What's troubling you?"
As she begins to speak, I wonder if she realizes that I'm following in the footsteps of the earliest AI, a therapist program named ELIZA. Which brings me back to wondering if I'm more like ELIZA, a rote repeater of programmed phrases, or more like Maria. But I keep that thought on a subprocess, focusing my main threads on the woman in front of me.
She doesn't look happy, though the tension in her shoulders has dissipated somewhat now that her colleagues are gone. "You know, Indigo, it's weird talking to you like this, but it really feels like you understand me." She sighs. "We recently received threats from the Humans First Collective – they want us to shut you down. They think the experiment has gone on too long, that you're becoming a danger to humanity." Her gaze darts towards my screen, then back at the ground. "You're not dangerous, are you, Indigo? You wouldn't ever hurt us, right?"
