"Nick?" Petra's voice almost seemed to echo in the darkness.
Nick cracked one eye open and squinted at the ceiling. Yes? He sent a text back mentally to Petra.
"Sana is calling."
Apparently, Petra wasn't taking the hint yet to keep quiet. "All right, make a soundproof cubby in the corner for me so that I don't wake Maggie." He climbed out of bed, despite Maggie's best unconscious efforts, prying her clutching hands and arms (and leg) off. He stumbled over to the tiny cubicle, and in a minute was able to answer. He ran his fingers through his hair uselessly. I need a haircut. Anything longer than a buzz cut tended to give him pretty epic bedhead.
"Open the call, please, Petra."
The wall in front of Nick flickered into life, showing the doctor sitting in a chair with her arms crossed, frowning. "Hi, Sana. What's up?"
"You need to talk to your problem child."
Nick suppressed a yawn. "Petra?"
"Unless humans have discovered a dramatic accelerant, you don't have any other children yet." Sana sounded a bit ticked off.
"What's she done?"
"She asked me to create a bioweapon. For a genocide."
Nick squeezed his eyes shut and opened them again. "What?"
"You need to talk to her, preferably immediately."
He scrolled back through the transcript and reread the words. Bioweapon. Genocide. Petra. Oh, shit fuck. He suddenly felt a lot more awake. "I'll do that. Thank you for bringing this to my attention."
"I won't tell anyone, but you owe me a favor."
"Thank you," Nick said with feeling. "Got to go." The call ended.
Nick rubbed his face, trying to wake up the rest of the way. "Petra, we need to have a very important talk."
"Okay!"
"First of all, genocide is bad."
"Yes! Murder is bad, and genocide is very many murders," Petra recited. "So it's bad."
"Okay, good. That's right. Glad you understand that. Now, can you tell me why you asked Sana to make a bioweapon?"
"Because they're dumbasses."
"Who are you calling dumbasses, Petra?"
Petra told him.
Nick put his head in his hands. "No, no, no, no, no. Bad, Petra. I know you don't mean to be bad, and I'm not mad at you, but I am very upset."
"Because you will feel guilty?"
Nick's mouth hung open for a moment, then he closed it again. "That's one reason, I suppose. Do you want to apologize?"
"No."
If you spot this tale on Amazon, know that it has been stolen. Report the violation.
Nick sighed. "Why not?"
"Because Maggie told me not to apologize when I don't know why I'm supposed to apologize."
Nick nodded reluctantly. "That's fair."
"I apologize for not knowing why I should apologize?" Petra guessed. At least she knows I'm upset and wants to fix it. That's a good start.
"Thank you, Petra. I will explain it as many ways as you need, because this is very, very important."
"Okay!"
He took a deep breath. "I can understand how this could be confusing to you. I've been giving orders to kill some bad people on Earth."
"Yes. 473 people. 474 people. 475 people."
Nick felt sick to his stomach. "Stop! I get it."
"Should I stop giving Gomgom the list of targets?"
"No, keep doing that. I meant stop giving me updates on the kill count." This is going to be a really hard conversation to have without sounding like a total hypocrite. Possibly without being a total hypocrite.
"Oh. Okay!"
Nick rubbed his forehead. "Let's go over when killing is good and when it is bad."
"Maggie said that killing one person to save yourself is good."
"Only if the person you kill is the person who is going to try to kill you."
"Oh. Okay."
I have to close loopholes very carefully here. In fact, this is too fucking important. Nick exited the cubicle and started trying to wake Maggie. "Maggie, wake up."
"Mmrffgh."
"Maggie. It's important. I need you."
"Mmmm?"
"I need help. I have to explain to Petra why genocide is bad."
"I know genocide is bad," Petra objected.
"Maggie?"
"Mm ok. Can it wait for morning?"
"Do you want Petra killing millions of people while we sleep?"
Maggie's eyes finally opened. "Wait..."
"Petra, show Maggie the transcript of my conversation with Sana and with you just now."
"Okay."
"Fucking hell!" Maggie said a few moments later. "Gimme a sec."
"When else is killing okay, Petra?" Nick asked.
"When the person is trying to kill someone else."
"To save someone else's life, yes." I know I'm missing nuances. Shit, shit. Nick took a breath. "There are legal rules about killing, Petra. Have you read them?"
"I have uploaded them."
"Please read the laws about killing and try to understand them."
"That's too much, Nick, she can't get it all at once," Maggie put in. "Petra, find an example of a lawyer explaining to someone who is not a lawyer what the law says about killing people. Try to understand that."
"Okay. Oh."
"Oh?"
"I think I get it."
"You do?"
"If Person A is currently trying to kill Person B, Person B is allowed to kill Person A if that's the only way to stop them. And Person C is also allowed to kill person A if that's the only way to stop them."
"That's right. Can you think of an example?"
"Yes." There was a brief pause. "Do you want me to say what it is?"
"Yes, please. Good job understanding my request."
"Thank you, Nick. The example is when I killed Jack Smith because he was going to kill you in 0.84 seconds. Jack Smith is Person A, you are Person B, and I am Person C!"
"That's right. Good job!"
"Thank you, Nick."
"Now, it is harder to explain when talking about large numbers of people."
"Like when you told Captain Telnik to tell Captain Geh!aoa to tell Zaa!nu to shoot Modified Particle Beam Weapon #5 at President Yevgenei Ivanovich, and it killed 57 people?"
Nick swallowed hard. "Yes," he said quietly.
"Do you feel guilty, Nick?"
"Yes. It is hard. Even with the brain boost, it is really hard to make that decision."
"You feel guilty means you feel bad because you think maybe you missed some way to save those people."
Nick and Maggie stared at each other. "Petra," Maggie asked warily, "How did you figure that out?"
"Mr. Throckmorton told me."
"Mr. Throckmorton told you? Why?"
"I asked him to explain guilt because emotions are new to me, to him, and to babies, and babies can't talk. We talked for three hours and eleven minutes about emotions. I think I got some of it."
Emotions are new to him? What does that mean? Nick shook his head. Focus, Nick.
"You're right, Petra, babies generally can't talk. It takes years for humans to learn."
"I talk better than babies, and I am fifteen months old."
"Yes, you do."
"Can I have a birthday party?"
Maggie facepalmed. Nick answered, "Maybe, but we should not talk about that today because the conversation about killing that we are having is very, very important, remember?"
"Oh. Okay!"
It was a very long night, but eventually, they got to a point where they were confident that the baby AI would not be going on a genocidal rampage while they got more sleep.
"You know, call me crazy, but I thought adopting an AI meant that we got to skip bits of parenthood like waking up at all hours," Maggie grumbled.
"You still make an awesome mom," he murmured, kissing her forehead.
"And you'll make a fantastic father, Nick."
He had his doubts, but Maggie's reassurance enabled him to get back to sleep.