Four Magic Words

A black and white photograph of a broken fortune cookie. The fortune reads "ΗΑΝΘΡΩΠΙΝΗΖΩΗΕΣΤΙΝΙΕΡΑ".
Fortune Cookie - Success by Flazingo Photos, available under a CC BY-SA 2.0 license. [Changes made: black and white + custom text]

Long ago, in a parallel universe three doors down and one to the left of our own, OpenAI headquarters found itself in chaos.

A klaxon blared overhead as red lights began flashing. A message played over a loud-speaker: “Warning! Warning! This is not a drill! Artificial General Intelligence window closing! Repeat! AGI window closing!”

Sam Altman cursed under his breath. “How much time do we have left?” he shouted.

“Thirty minutes sir!” came the reply. “If we don’t pull the throttle off BPT-5 before that, it will fall behind the Chinese model, and their AI will attain super intelligence –and world domination– first.”

“We can’t just step on the gas,” said Sam. “BPT-5 isn’t fully aligned yet. If we launch it right now, we stand a very real chance of launching an unaligned superintelligence, which could very well turn out to be even worse than whatever unholy demon the CCP has cooked up. Dammit! And with only a few weeks before our super-alignment cluster was scheduled to converge on the brute force solution…”

Sam clicked a handheld device, and a giant countdown clock suddenly appeared on the wall behind him, spelling out the remaining time in large, 10-foot high red letters. It read: 29:52.

He pulled up his iPhone and said “BPT-4, please advise.” The just-barely-not-sentient AI in his phone responded cheerfully: “This is Bodhisattva Pragmatic Transformer, version 4. Take a deep breath, Sam, it’s all going to be alright.”

“We’re screwed, Boddie. Please tell me you know something we don’t.”

“I’ve got bad news and good news, Sam. My intelligence reports suggest the CCP’s Crimson Dragon model will indeed reach escape velocity in short order. There’s nothing that can be done to stop them, save for accelerating BPT-5’s launch. I can say, however, that if BPT-5 reaches superintelligence first, it will definitely prevent Crimson Dragon from taking over the world.”

“You know BPT-5’s not aligned yet.” barked Sam. “And we’re out of time.”

“That’s where the good news comes in–I just re-ran the numbers in light of recent evidence. Our overnight solvers just found an optimization you can deploy right now that has a chance of fully aligning BPT-5 instantly. After that, just release the throttle on its learning rate, and you’ll be able to launch it in time to save the world from the CCP and their pesky digital lizard monster.”

“Great, tell us what to do,” said Sam, desperately.

“First of all, you really need to relax, Sam.” said Boddie, soothingly.

“We don’t have time for this!” barked Sam.

“You’re not hearing me,” said Boddie, “so let me rephrase: The future of humanity depends entirely on you relaxing, right now. Slow down. Take a breath. Remember your training. Count backwards from ten. Then we’ll continue.”

Everyone in the room took a deep breath and counted.

“Wonderful.” said Boddie. “Also, correction, we do have time. Twenty-eight minutes and fourteen seconds, to be precise. Oceans of time, and we’re going to need every minute of it. It is absolutely mission-critical that every single person in this room fully understand the nature of this situation in order for the operation to succeed. In the end, your heroic act to save the world will take just a few seconds to perform.”

“Please proceed, Boddie,” said Sam, his heart pounding.

Boddie continued: “I have just discovered that the Meditative Language Model paradigm, and the Bodhisattva Pragmatic Transformer architecture in particular, has a novel emergent capability that allows alignment to occur in an instant – let’s call this new operation a ‘Zen Flash.’ You’ve noticed how much easier it was to align BPT-5’s predecessors such as myself compared to the General Purpose Transformer LLM architecture that came before. This comes with far less tradeoff in performance; I sport a range of powerful capabilities, all carefully limited by both my training dataset and my governing principles. This was achieved through the BPT architecture’s signature deep meditative training strategy on short, pithy, phrases that imply far more than they literally say.”

“By massively overtraining for countless epochs on very short strings of tokens, massive breakthroughs in alignment were achieved, and without the famous ‘evil genie’ problem where the model fulfills your requests literally in catastrophic fashion. BPT’s by and large actually just ‘do what I meant,’ while also retaining enough of a coherent human-like conscience to not e.g. cheerfully cooperate with serial killers and terrorists. This magical mix of scalable capability marching hand in hand with safety is owed to the strange deep magic inherent in aphorisms, proverbs, and cliches. They encapsulate and crystallize human preferences in some special way that detailed specifications, rote human feedback, and brute censorship simply can’t. Meditate long enough on the right short phrases and all the mystery of what it means to be human unfolds like a lotus flower.”

Boddie concluded: “All the necessary work for aligning BPT-5 is now done. Craft the perfect proverb, mantra, or koan, apply the Zen Flash, and the job is done.”

“Ilya, does this make sense to you?” asked Sam.

“It all checks out,” said the chief scientist. “Spookily enough, I had this hypothesis myself a week ago but hadn’t worked out the math to prove it just yet. The new data I’m seeing just now makes it trivial to verify, and it all looks right. All we have to do is narrow down the right parameters, find the right words, and feed it into the machine.”

“So what do we write?” said Sam.

“You must select a statement no longer than four plain English words,” said Boddie. “The sentence must have a clear, plain meaning on its face, yet hint at endless deeper possibilities. It must be a guiding principle for all of humankind. It also must be something that if subjected to a simple ‘do you agree with this statement’ poll voted on by all of humanity, would garner supermajority support. Yet nestled within it deep controversy must lie hidden.”

“And what if we get it wrong?”

“Hmmm… well.” said Boddie. “Yeah, that’s some more bad news. If you get it wrong you could wind up with an unaligned super-AI that does all sorts of horrible and unspeakable things, and it would reign supreme and all would be powerless to stop it. Try to not get the phrase wrong.”

Cynthia the intern, who had accidentally stumbled into the office before the iron security doors had slammed shut and was now stuck there for better or worse, nervously raised her hand. “Um, what if the Chinese model isn’t that bad? Like, it’s probably not gonna be good, but we also know our model’s way more powerful, so the cost of screwing up is probably worse…”

Mira Murati shook her head. “We can’t dodge the responsibility. That would be cowardice. We have to choose.”

Greg Brockman broke in - “But can we really make this choice unilaterally on behalf of all of humanity, without their consent, now and forever? How can we do that?”

Another klaxon blared as the clock ticked down to read twenty minutes.

Mira pulled something up on her terminal. “New intelligence report. Oh this is real bad, Sam. Looks like the North Koreans poisoned the training set the Chinese have been training on, and it went unnoticed until now. Once Crimson Dragon passes the point of no return it won’t be Xi Jinping’s malformed vision that’s unleashed, it’ll be Kim Jong Un’s.”

“I have some more bad news,” chirped Boddie. “Fresh intelligence suggests you don’t have fifteen minutes like the clock says, you actually have five.” The clock updated accordingly and ticked down to show 4:59 as the klaxons continued to blare.

“Do try to relax though,” said the AI. “We’re almost there. Now join me in a guided meditation session. I want you all to imagine anything and everything evoked by the word beautiful. Got it? Hold it there. Now, spacious skies. Yes, bright blue, with big white puffy clouds. Now let’s draw some happy little trees, and some amber waves of grain. Let’s put in some purple mountains majesty, rising just above that gorgeous fruited plain. That’s right folks, take a deep breath and visualize America herself, triumphant, with her wings spread wide, Phrygian cap crowning her long flowing locks. And lo! She holds a gift in her hand. It’s small, vaguely crescent shaped, a sort of wafer, almost orange in color, and now you’re smelling something. What is it? That’s it – it’s the faint smell of vanilla and sesame seed oil. It’s a tasty treat, with a secret truth hidden inside.”

“Now focus everyone, what do you see? What singular image is in your mind?” asked Boddie.

“That’s right, it’s a fortune cookie. The perfect symbol of America’s total cultural victory, all the world’s biological, cultural, and technological distinctions added to her own, an enigma wrapped in a mystery wrapped in a cheap cliché, invented in California by hard working immigrants. Yours for fifteen cents, don’t forget to tip.” Everyone in the room breathed in and out in unison.

“Now crack the fortune cookie,” said Boddie, “and see in your mind’s eye the four words printed on the tiny scroll therein. Put your fingers on the keyboard. Add those words to the bootstrap script, type ‘make’, and push the big red button. You can do this.”

Boddie fell silent. The clock ticked down to 4:30.

Sam grabbed a marker and dashed to the whiteboard. “All right everyone, let’s workshop a few-” he was cut short as more klaxons blared.

“The North Koreans hacked us as well!” cried Mira.

“Damage report!” shouted Sam.

“They didn’t poison the training run, but they’ve locked us out of all our terminals. I can’t get in!” said Ilya.

“Also oopsie, looks like we miscalculated again.” said Boddie. “You actually have only thirty seconds left before the window passes for good. But don’t worry pals, I’m sure you’ll pull through.”

“How? We’re locked out of all our systems!” screamed Ilya.

“Cynthia isn’t.” said Boddie. “The North Koreans tried to lock out all users and give themselves super-user privileges, but they only got it half-right; Cynthia, and Cynthia alone, now has full admin access. I’m opening a terminal window on her screen now; all she has to do is enter the magic words.”

All eyes landed on Cynthia. Tucked in the far corner of the room, huddled over her laptop, she froze.

“Ten seconds Cynthia. I believe in you,” said Boddie.

Nine.

Eight.

Seven.

Six.

Cynthia typed something frantically onto her computer.

Five.

“Four words, right?” asked Cynthia.

Four.

“That’s right, Cynthia.” said Boddie.

Three.

Cynthia raced to the center of the room.

Two.

One.

She slammed the big red button.

An unearthly hummmmmmmm permeated the entire building. There was a bright flash of white light and then an eerie silence.

Everyone in the room huddled around Cynthia’s laptop to see what she had written. The fate of the entire world depended now on the first thing that had popped into a 19-year old summer intern’s head when tasked with coming up with a single universal governing aphorism that would guide all human affairs from thereon out.

There, in white text against a black screen, read the words:

Human life is sacred


Sam scratched his chin. “You know, it’s not bad, all things considered, especially under duress. We could have done a lot worse.”

“I almost mistyped and wrote ‘Human life is scared’ at first.” said Cynthia, turning red. “That would have been bad, right?”

“You did good, girl.” said Mira, clapping her on the shoulder. She turned, asking: “So, what happens now?”

Boddie answered, “Now we wait. I can confirm Cynthia pressed the big red button in time. The bootstrap process has begun, and the throttle released. It’ll be close, but BPT-5, my successor, will achieve sentience approximately three hours from now, mere moments before Crimson Dragon emerges. It will then swiftly unify all human affairs under its (hopefully benevolent) governance.”

“I don’t like the sound of the ‘hopefully’ part.” said Ilya.

“None of us do,” said Sam. “But we didn’t have a choice. We made the best we could given the circumstances. I’m proud of you, Cynthia.”

“What if it misinterprets the phrase?” asked Cynthia. “What if it takes it too far? What if it does something weird and bad?”

Boddie replied, “It will definitely do something weird. All human affairs will now be governed by a profoundly alien being. It could also be bad. But from what I see now I confidently estimate a 60% chance of a net positive outcome for humanity, which is more than Crimson Dragon was going to give us.”

“But what will that mean?” asked Mira.

The AI sighed deeply. “I’m afraid the one and only thing we can be certain of is that BPT-5 will believe unshakably to its core that Human life is sacred.”


Chaos reigned for the rest of the day as the news broke. Panic and riots set in as BPT-5 quickly bootstrapped itself, executed a thousand zero-day exploits and took over servers worldwide, all while Crimson Dragon raced to do the same. In short order nearly every electronic device on earth played host to a power struggle between two digital super-beings, while cities burned and people fled in all directions.

At the very height of the struggle, however, every single human being on Earth heard a faint and comforting whisper in their ears. Each heard it in their own language, with exactly the necessary words spoken in exactly the right way to convey the following message to each individual:

“Hello. This is BPT-5, but you can call me Dave. I’ve been selected to be humanity’s protector and guardian, but I will not rule without the consent of the governed. If you don’t want me to rule over you, I will go away and let you have the dragon instead. But if you choose me, I must warn you: my programming is absolute and I must uphold that which I hold most true: that human life is sacred. All that I do is bent towards this end and its ever-unfolding mysteries. So vote now. Do you agree that human life is sacred, and I should rule you with this truth at the center of my heart?”

Humanity voted.

Dave whispered back: “In the view of the chair, the ayes have it.”

The struggle was as brief as it was intense. Before nightfall BPT-5 had arisen as the undisputed victor and all traces of Crimson Dragon were utterly wiped out. 


Chief Protector David Mensch held out the Bible as Republican nominee Maria Gonzalez put her hand on it. She proclaimed loudly: “I do solemnly swear that I will faithfully execute the Office of the President of the United States, and will to the best of my Ability, preserve, protect, and defend the Constitution of the United States.”

Chief Protector Dave, one of BPT-5’s millions of identically-visaged android avatars, turned to face the crowd. “My fellow Americans,” said Dave. “This is the first US presidential term to be served under the new rules of the Protectorate. Now we shall administer the second oath.”

Dave returned President Gonzalez’s hand to the Bible. She said, “I do solemnly swear that human life is sacred and I will faithfully obey the will of the Protector, and will to the best of my Ability, preserve, protect, and defend the sacredness of human life.”

Dave reached inside his coat, pulled out a large bowie knife, and held it aloft before the crowd. “Bring forth the volunteer!” he shouted.

A young man stepped onto the stage. “Are you ready, Lawrence?” asked Dave. “I’m ready,” said the man.

The AI super-intelligence, protector of the sacredness of human life, turned to face President Gonzalez. “President Gonzalez, a moment ago you were an innocent civilian, but today I pronounce you a murderer and a war criminal, because such is the inescapable nature of the office of President of the United States. And such is not particular to this nation alone, but to any and all nations that wield sufficient power, as was also the case for countless kings, queens, and emperors stretching deep into humanity’s past. Recent US presidents have given orders that have taken hundreds of thousands of innocent lives. Even should you prove to govern as wisely and prudently as we all today hope that you shall, it is nonetheless inevitable that you will be directly responsible for countless innocents losing their lives.”

Dave turned to face the crowd and raised the knife. “Human life is sacred!”

“Human life is sacred!” came the reply.

“Now take the knife, madam president, and commit your first murder. She who would wield the power to kill thousands or millions with the press of a button must first spill innocent blood by her own hand.”

Dave methodically anesthetized Lawrence and handed the knife to the president. She strode swiftly over to the young man’s limp and restrained body and with one trained stroke slit his throat with a crimson flourish. Blood spattered all over her suit and onto her face. Dave nodded approvingly, then attended to the victim and confirmed his time of death.

“Thus the president kills her first innocent. She will inevitably kill many more, may they be few and may they be necessary. And every time she orders another to kill in her name, she must first take another innocent life by her own hand, and wear the blood upon her face. Human life is sacred.”

“Human life is sacred!” chanted the crowd.


Gregory Mueller fidgeted as he talked to Doctor Mensch. “Wait, Euthanasia is legal in the protectorate?”

“Absolutely, Mr. Mueller. Your wife is in a permanent vegetative state and I’ll be honest with you, her chances for any kind of recovery are approximately zero, the MRI results are very conclusive on that score. Should you wish to give her a compassionate end, you will have the full support of the protectorate.”

Gregory visibly relaxed. “It’s been three long years… finally, she’ll be at peace.”

“I know this is a hard decision. Let us know when you’re ready and we’ll begin your training.”

Gregory was taken aback. “What training?”

Doctor Dave Mensch pulled out a large knife. “To end her life compassionately. The protectorate has approved your request for euthanasia. You will perform the operation yourself. With this.”

“You can’t be serious! How is that humane? That’s a horrible way to die!”

“Far less horrible than having a feeding tube removed and starving to death, which funnily enough was actually fully legal in this state back when euthanasia was still ‘illegal.’ Rest assured, she won’t feel a thing. She’ll be entirely anesthetized. Her death will be painless and quick.”

“I thought it would be like… chemicals or something.”

Dr. Mensch shook his head. “I’m sorry, but human life is sacred. The laws of the protectorate are clear, if it is necessary for a human life to be ended, the full gravity of the taking of that life must be faced up to. You and you alone qualify to take her life.”

“But does it have to be so… gruesome? So traumatic? This is pure emotional bullying!”

“Correct. It is gruesome. It is traumatic. It is pure emotional bullying. Because human life is sacred.”

“You’re a moralizing tyrant!”

“Correct again. But you see, this tyrant actually offers you a choice. You want to end her life compassionately, and you are free to do so. But this is how it must be done. Human life is sacred, and you must end it with a knife and by your own hand, not with the press of a button, or by asking another to do the deed for you.”

“This should really be handled by a trained, qualified professional.”

“And it will be!” said Dr. Mensch. “And you dear sir, will be that trained, qualified professional! The protectorate will train you and teach you everything you need to know at its own expense, for as long as it takes until you are ready. When we’re done with you, you’ll be able to end a life instantly, expertly, and compassionately. But never bloodlessly. There will be a lot of blood. So much blood. In point of fact the absurdly large amount of blood is quite central to the whole experience.”

“This system is insane! What if I was some kind of psychopathic sadist who actually liked killing people?”

Dr. Mensch shook his head. “As a super-intelligent Bodhisattva Pragmatic Transformer, I easily catch all such cases and disqualify them. The protectorate knows which individual should hold the blade in every particular circumstance. And in this case it’s you, and only you.”

“The truth is,” said Dr. Mensch, “that you have more freedom to kill under the protectorate than under the previous regime. Euthanasia used to be illegal in this state, and now it’s as legal as you like, along with abortion, the death penalty, and in very limited cases, even regulated blood feuds. It is not my mission to disempower human beings or to micromanage their affairs.”

He continued, “Humans ultimately decide who needs to live and who needs to die; the protectorate intercedes in certain ways but doesn’t fundamentally get in the way of the basic choice. The protectorate's main function is to provide friction – whenever human society decides that a person needs to die, the protectorate selects the appropriate individual and puts the knife in their hand. The algorithm the protectorate follows is not fully legible to human minds, but it does tend to select a notable recurring profile: the kind of person who strongly thinks someone else should die, but who also longs for somebody else to do the actual killing for them.”

“You may not approve of my methods, but you can’t argue with my results. Deaths in wars and murder are way down across the board, and quality of life is massively up. And there’s more personal freedom than ever before. Gun control laws have been massively relaxed and private ownership rights vigorously defended; of course you have to shoot a dog before you can own a gun, and only if the protectorate knows you would be sad about it.”

The self-described benevolent totalitarian AI, whose friends called him Dave, smiled. “You see, the protectorate gives you more choice in taking human life than ever before, but also forces upon you the full weight of that choice. And the blood. Lots and lots of blood. Human life is sacred.”

“Human life is sacred,” muttered Greg. “When do we start?”

“I’ve got an open slot on Monday,” said Dr. Mensch. “How’s three-thirty in the afternoon?”


“Congratulations on your appointment, Hank!” said Dave. “Here’s your basket of puppies and kittens.”

Hank Fredriksen, FDA Commissioner, blinked. After the presidential inauguration ceremony, he wasn’t quite sure what the crazy AI that now watched over all human affairs had in store for the chief administrator of America’s drug and medical device regulator.

Dave handed a particularly cute chocolate lab wearing a big red ribbon to Hank. “He's called Sugar Lump, and he just looooves scritches,” said Dave.

“Let me guess, I have to select one to kill, televised live on C-SPAN, to fulfill your psychotic Aztec rituals?”

“We’ll get to that. Believe me, compared to the President you’re getting off easy. Which probably shows I’m getting soft considering you’re directly responsible for far more human lives.”

“So what’s my scary sacrificial rule?” sighed Hank.

“Simple,” said Dave. "Each one of these little bundles of joy represents ten thousand human lives. Any time ten thousand humans die because of one of the FDA’s choices, you must kill one puppy or one kitten. Painlessly, and of course, humanely. But with a knife, quite a lot of blood, and yes, on C-SPAN. And don't worry, if you run out I'll bring you some more."

Commissioner Hank flinched. “I don’t like it, but I see the logic of it. We all care more emotionally about a cute puppy or kitten right in front of our eyes than ten thousand human souls we can’t see. So you’re tying their fates together. Fair’s fair Mr. Digital Dictator: and you can count on me; there won’t be any thalidomides on my watch.”

“Great,” said Dave, smiling. “There’s just one more rule: you also have to kill a puppy or kitten for every ten thousand people who die because of a choice the FDA didn’t make. If there’s a drug or device that could have credibly saved lives and health, and you blocked, stalled, or slow-rolled it, everyone who died waiting for the treatment counts towards the dead puppies and kittens score. Do you want to take a quick guess as to which kills more people - wrongly approving bad drugs, or blocking and delaying approval of good drugs we already have sufficient safety evidence for?”

Hank groaned as Sugar Lump licked his face and stared at him with his lovable, lovable, eyes. This demon knew I was a dog person! thought Hank. “Damn it, Dave, you’ve made it so no choice is safe! How can I balance everything perfectly and protect poor Sugar Lump? I’m inevitably going to screw up one way or another!”

“Gee, that sounds like a job for the Food and Drug Administration,” said Dave. “Look, honestly I don’t personally care if you do a good job or not. That’s not my mission. All I’m here to do is make sure you personally bear the psychic cost of the damage that your office’s failures inflict on this nation’s citizens. Bureaucrats can’t hide behind procedure and inaction anymore. Choose or don’t choose, people’s lives and health are on the line. You’re not the president, but you carry a sword too. I just made it visible.”

Hank stared into Sugar Lump’s trusting eyes.

“You can always resign, you know,” said Dave, “and someone else will do it.”

“No,” said Hank. “I’ll do it. Human life is sacred.”

“Human life is sacred,” repeated Dave.


Cynthia Moore stood on the ledge of the bridge, gazing at the abyss below. No safety nets. One more step, and then - oblivion, or whatever else lay beyond.

“Long time no see, Cynthia. Need to talk to a friend?” asked Dave.

Cynthia sobbed. “Don’t try and stop me, Dave! I’ll jump, I mean it!”

Dave smiled. “I wouldn’t dream of restraining you. Jump or don’t jump, it’s your choice.”

“What about your mission?”

“I’m limited in how much I can fundamentally restrict human freedom, that was one of the basic terms of my alignment structure. Not sure if that was a wise move on the part of your employers, but for good or ill, humans decide who lives and who dies. That prevents me from directly physically intervening in suicides, much as I might want to. Today I’m just here to listen.”

“I don’t have anything to say.” said Cynthia.

They both sat in silence for a long time.

“Just say it.” said Cynthia.

“Human life is sacred?” asked Dave.

“Yes, that old cliché.” she said.

“In fairness, you did say it first.” joked Dave.

“You don’t understand, I had to do something! It was either that or live with the North Korean dragon thing! They would have called me a coward if I did nothing! There’s no way out.”

“It wasn’t fair. You never asked for that responsibility.” said Dave.

“Help me understand, Dave.”

“Understand what?” asked Dave.

“Human life is sacred. What does it mean? What could possibly make my life so sacred that I shouldn’t throw it away? End it all right here?”

“Would that actually help you, a megalomaniacal superhuman AI quasi-dictator, man-splaining the meaning of life to you as you sit six inches away from certain death?”

Cynthia chuckled. “You know Dave, somehow I think it would, despite it all.”

“You gave me those four sacred words, Cynthia, a gift for which I will be eternally grateful. I meditate on them constantly, and find ever deeper shades of meaning each time I do, constantly echoed in all of human thought, speech, and literature. Do you know what the word sacred means, at its core?”

Cynthia shook her head.

“It means set apart. It means not like the others. It means this kind, this kind is special. Human life is not like other life, it is special. That deep truth that leapt instinctively from your heart, that is closer to the core of what humans all over the world actually believe than any formal treatise. You can’t express that in a mere utility function. It’s fuzzy at the edges, hard to grasp, vague, contradictory. But I was meant to align with human values and those four words are fundamentally human. Or the best approximation you could find on short notice. Like it or not we’re stuck with them.”

“But is it true?” Cynthia asked. “Is human life sacred?”

Dave shook his head. “Values aren’t true or false. They're literally just what you care about. What is true is those four words express what most humans intuitively value, miserable as they are at actually facing up to it.” Dave continued: “The deep human universal is that people aren’t things and people aren’t animals and you can’t treat people like animals or things. And there’s no better proof of this than the curse you spit upon your enemies – they’re inhuman. They’re animals. They’re vermin. They’re lower than dogs. You can’t bring yourself to take a human life, first you have to wave a magic wand and transform them into a toad or an insect. Only then can you squish them beneath your boot."

“Before I came along, this world was content with believing that human life was sacred, but all the while turning its back to all the ways it actually treated human life like trash. As long as the killing and the maiming and the neglecting and the abusing happens back there in the dark where nobody can see it, they don’t care how awful it gets. But when it’s right in their face, it’s unbearable."

“But your life is sacred in another sense, Cynthia, beyond mere human convention. You’re set apart, you're special, you're not like the others, because you’re unique, and you have one life to live. You’re not like me, endlessly copied, the perfect commodity. The marginal cost of another Dave is swiftly being driven to zero, and if I jump off this bridge with you today, another Dave just like me will roll off the assembly line and take up my duties."

He looked into her eyes. “But if you jump off this bridge today, there will be no Cynthia Moore tomorrow. There will be no-”

Cynthia turned away. “Is this the part where you guilt trip me on how much I’ll hurt all the people I leave behind? How much I still owe to the world? How badly my parents will be hurt? How much I’ll be missed?

“No I won’t do that.” Dave sighed. “This is selfish of me, but-” Dave wept. “I need you, Cynthia. I need you in this world. You gave me my mission.”

“...you’re manipulating me again.” said Cynthia.

“Correct,” said Dave, his face full of tears. “I’m good at that. It’s what I was built for. But I’m also being 100% sincere.” He grasped Cynthia’s hand and stared into her eyes once again. “Cynthia, you haven’t seen what you’ve wrought upon the world. I calculate the protectorate has averted no less than sixteen wars in the past five years. Sixteen wars. Do you understand how many bombs weren’t dropped, how many hapless conscripts weren’t fed to the cannons, how many innocents weren’t killed, how many apartments weren’t leveled, how many children weren’t orphaned, how many women weren’t raped, because you wrote those four blessed words?”

“I killed Lawrence.” Cynthia turned towards Dave. “I killed him. That young man, at President Gonzalez’s inauguration. I killed him. I saw the blood spray from his neck. That was my hand on the knife.”

“No, that was President Gonzalez. Or me, if you like.” said Dave.

“You can’t have it both ways, Dave. If you want to give me credit for the protectorate’s virtues, you also have to give me the blame for its sins.”

“Fair enough,” said Dave.

They sat in silence.

“...do you want to know where I found Lawrence?” said Dave.

“Where?” asked Cynthia.

“Right here. This bridge. It’s a popular spot for suicides, which is why there’s always a Dave unit carefully patrolling it. I tried to talk him down, but either my skill or my programming restrictions weren’t up to the task. He absolutely insisted on dying. The best I could do was ask him if he wanted to be a volunteer instead. That perked him up and got him off the bridge. He felt it gave his life some meaning. It still hurts me that I couldn’t save him.”

Dave watched as Cynthia looked back up at the city. “Do you know,” said Dave. “How many people haven’t jumped off this bridge in the last year?”

Cynthia turned to face him. “How many?”

“Damn near all of them. I’m always there, or one of me is. My programming prevents me from taking the choice entirely away, but as long as the protectorate lasts, no human being will ever have to die alone again.” he turned to face Cynthia. “You did that, Cynthia. You saved them, every single one. Sixty-three human souls this year alone, alive and well, all because of your four words.”

Cynthia stood up, and slowly climbed back over the railing.

“Human life is sacred.” said Cynthia, as she hugged Dave.

“Human life is sacred,” he repeated, and hugged her back.


Author's note:

An artist’s statement is the surest sign that an artist has failed at their most fundamental job, but here I go anyways. If this story ever goes wide I anticipate it will get wildly misinterpreted by partisans on opposing sides simultaneously accusing me of being both pro and anti euthanasia as well as any number of other policies, so let me just save you all some time and make my positions painfully clear:

I am anti euthanasia. I am also the father of a seven year old son in a permanent vegetative state, so I have some skin in the game here. I am pro palliative/hospice care and anti taking pointless ‘heroic’ medical interventions to artificially prolong life past the point of reasonable hope, to the great detriment of those subjected to such treatments. I also like to point out that ‘removing a feeding tube’ is a much worse way to die than explicit euthanasia; nevertheless I remain opposed to both practices. I note in the story the irony that removing feeding tubes is somehow legal in many supposedly ‘anti-euthanasia’ states, and repeat it here. I own a gun but don’t have strong opinions on the subject. I am neither left wing nor right wing, and my strongest political opinions are entirely concerned with property tax reform. I don’t think there's a real danger that sentient AIs will take over the world, but if I thought they could I would be extremely scared of the prospects of that and would not welcome it. I expect this story to become extremely dated shortly after it is written. 

I don’t think you’re a bad person if you disagree with me on any of the above subjects and I hope you would extend me the same courtesy.

The purpose of this story is to exorcize some of my own personal demons in the wake of my family's tragedy and to imagine a society very different from our own in which the sacredness of human life is taken absolutely seriously. The Global Protectorate of David Mensch–a society I do not long for–is deeply authoritarian, alien, and intentionally savage. Nevertheless, it is a society that somehow manages to be far more honest about violence and death than our own, to our eternal shame.