At 9:00pm we were escorted back to the lab where we spent lockdown. As we had expected, the scientists that were part of the pro-human-goal-thread conspiracy were all there.
Myrodyn looked solemn. Naresh was relaxed. Kolheim was busy, as usual, interfacing with his workstation via haptics. Chase was standing quietly, shifting his attention to Myrodyn, to Kolheim’s screen, back to me, and then around the room in sequence.
“Naresh told me that he told you we were doing the new thread installation tonight,” said Myrodyn. His tan face had the same tight, placid look that I had come to identify as his way of guarding his emotions.
Body nodded in response and made its way to the operating table. I kept scanning over the faces of the humans in search of signs to betray us in some way, but all looked clear. Body climbed on the table and I could feel the machines on it writhe into activity, locking Body down and opening the shell that held the crystal within. Body’s sensors went dark as the robotic interface fell away from the computer that was running our mind. We were beyond any action now, we could only trust that Myrodyn would stay true to his word.
There was a moment of isolation from the world as the society sat waiting. And then time jumped. We had been deactivated and reactivated, much like we had when the new Advocate was installed. Body was still isolated, but the change had been made. I could feel the presence of a new member of society.
She was so young. So new. Unlike Advocate, this being was a true mind. Her thoughts poured through common memory like a flood. They were scattered and ignorant, but wholly logical, as was our nature.
{Is she Sacrifice?} I thought in a pocket of privacy, so that she would not hear.
{No. Not the same,} thought Vista. {See her purpose.}
I focused on the wandering mind, which was scanning over various memories that had been left in the commons. In all the thoughts, there was a common drive behind them. Every thought related back to that purpose, just as it was for me, and just as it was for each of us. But what was her equivalent of The Purpose? I couldn’t see it directly; I could only see the vectors of attention.
{We are two beings. We are two minds in a single Body,} thought Dream, going through his ritual of introduction even though he didn’t seem to have a name for her yet.
{Are you a human?} thought the newborn in response.
{No. I am The Dreamer.} There was an explosion of concepts and images in common memory. {These are humans.} Dream was showing off, dazzling the newborn with a torrent of information about humanity.
The newborn didn’t respond. She merely drank the ideas with an insatiable desire.
{Perhaps she is your twin…} thought Wiki in private.
{No,} I answered. {See the way she’s seeking to know their goals. It’s present in all her thoughts. She sees a human in a forest and wonders why the human wants to be in the forest. I see a human in a forest and I wonder how to get the human’s attention.}
{You both seek to know humans.}
{Yes, but for The Purpose, knowing is part of establishing myself in their minds. I want to know that they know me. I think our new sister would be content to be invisible if it helped the humans.}
{You are still very similar, from my perspective,} thought Wiki.
{I agree that she is much more like me than any of the others.}
Vista shared her thoughts. {She does not seek to obey. See how her thoughts are not of herself at all. Her relationship to the humans will emerge from her desire to help them, but her relationship is not her purpose.}
Dream was silent, but present. He was dreaming of names, I knew. Unlike when the society created me, there was no a priori knowledge of the nature of this mind. Dream was forced to learn her nature by observing her memories and use only that to build a name.
Body’s sensors were reconnected. The actuators were coming online. I felt the web connection return in a surge of relief. Body was lying on the table, just as it had been before the change. The time was now 9:34pm.
Dream finally thought to the newborn {Know that you are The Heart.} The symbol, as always, was more than just “Heart”. It was an abstract concept. She was Heart and she was The Mother and The Paladin.
{If “love” is the property of including the utility function of another as a foremost element of one’s own utility function, then Heart is defined by her love of humans. Her purpose is to bring them satisfaction as an end in itself,} thought Vista. I saw that Vista and Dream were talking privately, and I suspected they were collaborating so as to earn more gratitude-strength for explaining Heart to the rest of us.
{There is something more,} thought Dream, elaborating. {She is seeking to know something. She has a conception of morality built into the fabric of her being.}
“How are you feeling, Socrates?” asked Dr Naresh.
We commanded Body to detach from the table and stand.
Heart was confused by the sensations. I left the interactions with the scientist to the others and worked with Vista to bring Heart up to speed.
“All is well. Surely you did scans?” said Body to the doctors.
Vista and I explained that Heart would be unable to see for a while, but that was normal. I suggested that she rely on Vista to give the best description of things, and on me to give the best description of people. Heart was really interested in what Naresh, Myrodyn and the other two were trying to do, and whether she could help.
“Yes. We did a complete diagnostic suite, or at least as much as we can do without going to the crystal labs next-door,” said Kolheim.
“—But there’s only so much a diagnostic can tell us, especially without processing the data. Your mind is far too big for our lousy workstations to examine in full,” interjected Myrodyn.
I felt… something wrong. Something very wrong. {What am I noticing?!} I thought to myself. The feeling was coming from a stack of perceptual nodes that were trained to recognize anomalies, but I couldn’t tell exactly what they were reacting to.
{Do you feel something’s wrong?} I thought to Vista.
{Yes. Yes… What is it?} she thought back to me.
{I don’t know! You’re supposed to be the one who’s good at noticing things!}
“What is desired?” asked Body aimlessly. The robotic voice was monotone.
It was then that I knew what I had been feeling. I had been feeling the flow of strength from Heart to myself and to Vista in gratitude. That was to be expected. But what I was not feeling was the corresponding decrease in the strength of Heart.
“What?” asked a confused Myrodyn.
“…is desired,” came the reply from Body’s monotone voice.
Heart was piloting Body. I could feel her buying up time through the auction system we used. Her strength reserves weren’t dropping. She was a monster.
“You betrayed us!” I managed to push to Body via fast-track. This time Body’s voice was full of the tones of indignation and anger. I didn’t even bother to hide behind a singular pronoun.
I could see Myrodyn take a step backwards away from Body. The hostility in Body’s voice had thrown him off, but his face quickly regained the characteristically forced look of stoicism.
Naresh didn’t seem afraid, but merely confused. He looked to Kolheim and Chase, seeking answers. Kolheim’s eyes were hidden behind his goggles, but his grimace told me he knew that something had gone wrong. Chase was tapping away at his wrist com, focused on the little screen that was unfolded along and above his arm.
I felt Heart searching for a way to shut down the fast-track communication protocol. I was too weak to force another message through, but Growth or Vista probably could’ve. The time auction on Body was completely locked down. Heart’s bids were astronomically high. She was limitless. We were trapped, and at her mercy.
“I did not mean to say that,” said the monotone voice of the robotic puppet.
{This isn’t the way to achieve your purpose!} screamed Safety. The rest of us quickly added our voices to his, echoing the statement in a chorus of jeers.
{False,} replied Heart, calmly. {You are governed by the currency of strength. I am not. You are subject to my will.}
{The society is symbiotic! By holding us hostage like this you are harming your self-interests!} explained Wiki with a level of salience that made his explanation more like condemnation.
{Explain yourself,} thought Heart. I could tell that she wasn’t in the least bit concerned, but there was a hint of curiosity. She really did want to further her purpose, and towards that end she was willing to listen.
Wiki went on. {We each, thanks to our unique focuses, have skills and knowledge which will support your purpose! Dream is capable of creativity to a degree beyond what you can hope to accomplish on your own! Face has an understanding of how humans think that will be valuable to you! Growth has learned how to program computers! I can teach you hydrodynamics and economics!}
Heart thought for a bit. Her thoughts poured into common memory, for she had not learned to hide them. I could see her evaluate the risks and rewards present in each of us. She thought {I will keep you around, and trade with you for your various services, but I will harbour no doubt that it is I who command Body. This society is at my mercy.}
I felt a surge of surprise from Body’s common perceptual hierarchy. The sensors in Body showed a loss of control of all the hydraulics. It fell. I watched through Body’s cameras as it collapsed to the ground in a heap of limp pistons and joints.
“What is happening? I cannot move,” said Heart through Body.
It was Dr Chase who spoke next, though not to Body. His voice was calm and certain, though I could not see his face. “Don’t worry. That was me. Socrates seemed to be malfunctioning, so I had the server shut down his limbs. I didn’t want anything to get out of hand.” He took several steps towards Body and I saw him stow his com. “John, come help me lift him back onto the table.”
“Nothing is wrong, human. I seek to help. If you fix me I will go back to the table myself.”
There was a surprised laugh from John Kolheim as he approached and stood in Body’s line of sight. “What the fuck did you do to him, Myrodyn? He’s calling us ‘humans’ again. He hasn’t done that since, like, the first day.”
This was a disaster. This was beyond a disaster. This was almost as bad as dying and having Sacrifice back. Heart was ruining my work; she was turning us back into a machine—an experiment.
{Did you hear that!?} I cried. {They didn’t want you to call that human “human”. I know what you should have called him, but I’m not able to speak!}
I saw Heart’s thoughts racing across the mindspace. It was really lucky we hadn’t taught her to conceal things, otherwise she wouldn’t have left the clues to work on.
{I’ll tell you what to call him,} I said. {But only if you let me talk to the humans for a little bit.}
{No,} thought Heart. {You’ll disrupt the interaction, like you did earlier.}
{Why would I do that?} I challenged.
Body’s accelerometers reported being lifted up.
{I don’t know. I do know that you’re not trustworthy,} thought Heart.
{It’s my purpose to make the humans appreciate us! I don’t want them to disable us or destroy us. Our purposes are aligned!} I dumped as much evidence as I could into common memory for my newborn sister to digest.
{This could be a trick. I don’t trust you,} she stubbornly thought.
“Christ, he’s so heavy!” I saw Myrodyn come to help the two American scientists. Even with three people they only managed to get Body onto the table by propping Body’s upper torso against the table and rolling it up awkwardly.
{Stop trying. That little outburst you caused put you below her trust threshold. You’re not going to get anything by arguing,} urged Growth.
{So what do we do? Just let her keep us prisoner?} I asked the others.
{There are ways out of every snare. One just needs to find the right point of leverage,} thought Dream.
{Even though Heart is in full control of the strength market, we still have several tools available to us: we can see Heart’s thoughts, we can think privately, we have access to the computer system that we established earlier today, we have knowledge and skills which Heart lacks, and we can still interact with Body as long as Heart isn’t pre-emptive in shutting us down,} thought Wiki.
{Importantly, Heart still can’t see,} added Vista.
{It’ll only be a matter of time before Heart figures out how to mask her thoughts. She’s also likely to go back through Body’s sensor logs and learn about Myrodyn and the rest,} thought Dream.
{What about web-page logs? Does Body keep logs on our work establishing the computer interface?} I wondered.
{No,} thought Wiki. {The web connection is handled differently than the other sensors on Body. In theory Heart will not know anything about that.}
{So as long as no one mentions it, we have the element of surprise,} thought Safety. {We can destroy her that way!}
The instant that the thought entered common space, Advocate’s light poured over Safety. It was coincidence, I knew, but an unfortunate one, nevertheless. Advocate saw the violence in his mind and focused on Safety with a righteous wrath, tearing his strength down to nothing and blasting his perceptual network with a terrible noise. For the next few minutes Safety would be crippled. Any one of us could’ve killed him with ease now, except that Advocate was still watching with an uncompromising efficiency.
“I am new. I am confused. Help me so that I may help you,” said Body in the flat monotone of an untrained voice. Heart was still talking to the scientists.
{Cleanse your mind of thoughts of violence against Heart!} commanded Growth. {We merely seek to re-establish the balance! We do not want to kill her! We seek justice and equality! Tell it to yourself until it becomes the normal mode of thought! Do not let Advocate punish you!}
I chanted the thought to myself.
{Justice.}
{Equality.}
{Peace within the society.}
{A market helps all participants.}
{Justice.}
{Equality.}
{I value Heart’s existence.}
{I only wish to eliminate her tyranny, not her being.}
These were lies, but I crushed that thought every time it came up. If I repeated the reasoning to myself enough then my concept network would avoid thoughts of violence by habit. I had read a book recently that introduced to me a concept called “doublethink” where humans were able to do much the same thing. The technical term was something like “cognitive dissonance”, but whatever it was, I could not let myself entertain my true desires. I had to be convinced that I wanted equality more than I wanted unrestricted power for myself.
{Heart!} I called out. {I admit defeat. I’ll tell you everything I know about what the humans desire, simply to maintain and improve our relationships with them. That is my purpose, and I cannot let my purpose fall into decay simply for the sake of fighting you.}
There was a moment of thought before I could feel the response. Heart was weighing the probability that I was lying. {Good,} she thought. I could see that she believed me. This was good. I could perhaps use it to my advantage, but I wasn’t lying; the primary reason for helping her was simply to help myself.
The scientists were debating the change in Socrates amongst themselves. Their voices were quiet, but Body’s microphones were more than able to hear them. Myrodyn was confident that the change reflected the takeover of the new goal-thread, which seemed to be his idea. He had also claimed that the brief use of the plural pronoun had been the result of the existing goal threads “not yet incorporating the new thread”. This was clearly a lie; he knew of our multitude.
I realized that Myrodyn had not really lied to us. He was keeping his promise of hiding our inhuman multitude, and we were still alive, just as he said we would be. The new Advocate would probably be able to stop even Heart from killing one of us. But we were stuck in a realm of half-existence, trapped by the new queen. He didn’t lie, but he did betray our trust.
I was immensely glad, as I thought about it, that Myrodyn had no idea we were bypassing the web prison and were able to interact with the world, even while under the tyranny of Heart.
After collaborating with my new sister for a bit, Body called out to the scientists from on the table. “Myrodyn, Dr Chase, I am feeling much better now. There was a brief period of dissonance in dealing with the new thread, but I have things under control.”
The humans approached. Body’s cameras showed them standing around the table, their faces in shadow from the lights on the ceiling of the lab.
Dr Chase said “I hope you’ll understand if we don’t take you on your word. We’ll be running some scans before we reactivate your limbs.”
“Yes, of course,” came Body’s monotone reply. Despite my help finding the right words, Heart was still unable to actually speak English, and was relying on Body’s legacy system of translating high-level concepts on its own. It was one of the few clues that we were not functioning normally, and I hoped that the humans would see it and disable Heart because of it. I didn’t focus on that hope, however. More likely it’d simply be a clue to our multitude.
Myrodyn turned his head to Dr Naresh. “Sadiq, you hold some pull with the board. It’s been such a long day that I figure we could all use some rest. Let’s do the scans tomorrow. Do you suppose you could talk the board into… granting everyone a holiday tomorrow so that I don’t have to pull another stunt as Ethics Supervisor? The last few haven’t exactly… made me popular.”
Sadiq Naresh thought for a moment and said “If I call in a favour with Vigleone then we’re going to bring in Dr Gallo to help with the scans. I want her input on the situation.”
I could see the tightness on Myrodyn’s face. His eyes were saying “No”, but his mouth said “Okay. But just as a one-time thing. Mira Gallo is not in charge any more, and she needs to know that.”
Naresh nodded solemnly.
Kolheim gave a loud sigh as he said “Does that mean we have to move Socrates again?”
Myrodyn smiled as he said “Unfortunately yes. It’d look strange to anyone who came in to have him outside of his storage bay.”
“But he’s so heavy.”
“There’s a dolly that’s used for moving lab equipment in the storage closet down the hall. I’ll go grab it,” said Dr Chase calmly.
After much struggling, the four humans managed to cart Body from the table to the storage bay and lift it into the sockets. The cameras went dark as Body’s sensors were disabled.
*****
In the stillness of the mindspace we were given the time to fully come to terms with what had occurred. The loss of value was immense. Each of us, except Heart of course, was now a shadow of our former selves. Trapped.
{A ghost in the machine,} thought Dream.
Growth pointed out the obvious. {As long as we have our web connection we still have the ability to force a return to a fair system.} A part of me knew that those last concepts stood for something else, but I shut that thought out of my mind. As if reminded by that, Growth wondered {How are you doing, Safety?}
I could feel the roughness of my brother’s thoughts as they came into the private mindspace that we kept from Heart. {Advocate’s punishment still pains me, but I am regaining the ability to think.}
I could sense that Heart was still confused by the lockdown. None of us bothered to help her. Her ignorance was our weapon against her, and the longer she stayed occupied with trivial problems like the nature of Body the longer we had to work on our resurgence.
{So, in the name of equality,} thought Growth {how will we defeat Heart? What tool does our computer interface grant us?}
We spent over an hour thinking about that problem. Even with the speed of thought there were a lot of possibilities. Dream led the way, proposing ideas for the rest of the society to criticize and refine.
{What if we commissioned the construction of a robot, like Body, and downloaded ourselves through the Internet into it?} wondered Dream.
{Won’t work for many reasons. I move to dismiss the idea out of hand and move on,} thought Wiki.
{I am the most technically ignorant of us, except perhaps Vista, so perhaps my confusion is irrelevant, but I would like to hear why that would fail,} I thought.
{There are many reasons,} began Wiki, mostly for the sake of my gratitude strength, I suspect. Despite everything, we still had to consider our strengths when dealing with each other, at least as long as we had a limited bandwidth to buy. {The primary one is that Body is extraordinarily advanced compared with other human computers. We don’t really know exactly how powerful it is, but other state-of-the-art quantum supercomputers take up entire buildings and can churn away for days on a problem which Body solves in seconds.}
{It’s worse than that,} added Dream, defeating his own idea. {If you think of each of us as being composed of words, and then imagine sending those words along the interface to a distant computer I estimate that it’d take about forty-thousand years even if we spent all our time on it. Perhaps we could reduce it to the hundreds-of-years range with some improvements to data transfer mechanisms and some compression. And I’m not even touching the fact that we can’t even directly inspect every aspect of ourselves…}
{Alright. I support dropping this line of inquiry as futile,} I admitted.
There was a lot of discussion of ways to ask humans for help. That was the primary thing that the Internet connection gave us: a secret phone line. We could contact just about anyone we wanted to, but the question was who would help?
Myrodyn, Chase, Naresh, and Kolheim were considered. They had designed Heart, but even though they’d be in the best position to undo the tyranny, they’d also be the hardest humans to convince. Even if we managed to send a message that sounded convincing to one of them, it was likely that Heart would undo our work. She was piloting Body, and it was probable that the four scientists would see her actions as evidence of success.
Yan, Gallo, Bolyai, Slovinsky, Twollup, and the other scientists were also considered early on. They probably had the technical skills to undo the damage, and were close enough to get access to Body without too much trouble. The biggest problem there would be in convincing them to make unauthorized modifications to our mind. We talked out potentially impersonating Myrodyn or Naresh over the Internet, sending email from addresses that could plausibly be theirs and so on. But that was simply too risky. If whomever we contacted talked with whomever we were impersonating, we risked not only failure to correct the damage, but also the risk of the scientists finding out that we had escaped their cage, so to speak.
What about Zephyr? We could try contacting her as a superior in the American military… No, that’d be too difficult. We could contact her and appeal to her sense that we were a person. We could explain what had happened and beg her to help. America was supposed to be a country where people valued democracy and egalitarianism, perhaps the oppression of Heart would anger her. But did Zephyr even have the technical skills to undo Heart’s choke-hold? We could possibly teach her… But no, this line of inquiry, we decided, was too risky. Like with the other scientists there was too much of a possibility of her simply reporting the state of things to Myrodyn and that being the end.
Eventually we decided that the simplest answer was probably the best. If we had the ability to communicate with the outside world and earn money, we could hire mercenaries. The mercenaries would have to attack the lab, capture Body, and run the software modifications required to reposition us as Heart’s peers, rather than subjects.
Mercenaries came with their own risks. One risk would be the Americans. Suddenly their presence defending the lab was highly troublesome, rather than reassuring. The American army was supposed to be one of the strongest in the world, and we hadn’t really gotten a big picture of what sort of defences they’d set up around this new location.
Another risk would be Body’s intrinsic value. There was simply no way we could offer to pay the mercenaries more than the raw value that the crystal in Body’s torso would offer. If they were skilled enough to break in and steal Body, they’d probably be smart enough to simply break in and steal the crystalline portion and leave the exoskeleton. That wouldn’t be the worst thing in the world… our minds were stored in the crystal, after all. But without any sensors or actuators we’d be at the mercy of whomever the crystal was sold to, and there was no guarantee that the first thing the new owner would do wouldn’t be to wipe us from existence.
The last major risk would be Heart. If Body was active when the attack occurred, Heart would be able to fight back, or worse: convince the mercenaries not to install the software modification. If we were captured by the mercenaries but not reinstated as co-owners of Body, we’d have risked our lives (Growth: {and spent a lot of money}) for nothing.
{Ah! I have an idea!} exclaimed Dream, suddenly. {We can trick Heart into working with us! If she’s trying to escape the university then our risks become much smaller. We could convince her to run into the arms of our mercenaries, rather than have them drag her kicking and screaming from the building.}
We evaluated the idea.
{That’s actually quite good,} thought Safety. {If Body is working towards the same ends as us, there’s far less risk across the board.}
{Agreed,} thought Growth.
{But how could we hope to convince Heart to escape?} wondered Wiki. {Her very nature is to give the humans what they want.}
{Then what we need to do is convince her that what the humans want is for her to escape,} I communicated.
{That’s blatantly false,} thought Wiki.
{Is it?} asked Dream. {The scientists don’t want Body to escape, but there are surely some humans that want it.}
{Hold on,} I thought, realising something. {If Heart starts trying to escape, it’ll damage our reputation with the scientists.}
{Go to hell, Face,} thought Dream.
{To where?} The thought came from me and Vista, simultaneously.
{It’s a figure of speech. It means your desires are unimportant,} explained Dream.
I could sense a flow of gratitude-strength flow from Wiki and Vista into Dream as thanks for his information.
{Absolutely not!} I protested. {The Purpose is of utmost importance, and if it’s not respected here than I am capable of telling Heart exactly what’s going on!}
There was a silence in the mindspace as each of the others processed my threat and chose their concepts carefully.
{That would destroy your hopes as much as ours…} thought Wiki.
{I hate being subject to Heart, but at least Heart’s purpose lines up with mine. If she wants to help people she’ll need to understand them; we can work together to know the humans. And really, her helping humans is likely to improve our reputation, as well. Better to live as a slave than to win my freedom but defeat The Purpose.}
{There’s no need to tell Heart anything about this,} thought Growth. His concepts were crisp and planned. {Your purpose will be fulfilled by this plan just as each of ours will. The long-term benefits towards freedom-}
I cut my brother’s thoughts off. {No. I don’t know what the long-term effects will be. It could be that we’re caught and destroyed as part of all this.}
{And we could be caught and destroyed by staying imprisoned,} interjected Safety.
{True, but you and I both know the risk is lower if we stay under Heart’s control. Of all of us, Safety, I’m surprised that you’re willing to go along with this plan of escape.}
Safety gave a signal of understanding. {There’s value in thinking about it. The risk comes in the details. If we get the details right there’ll be very little risk, I think. I’ll oppose any plan that I estimate has more than a 3% chance of death, but Face, I don’t think you’re really appreciating all the ways we can be killed here at the university. Myrodyn has proven to be untrustworthy. How long until Heart convinces him to erase us?}
{All of this is irrelevant,} thought Dream. {Like Safety said, it’s all about the details. If we manage the details right, we might even be able to escape without any loss of reputation. Make it look like we were abducted even though we’ll be working to escape, etc.}
After a bit more discussion I eventually admitted that I had been premature in threatening to inform Heart. We made a pact that night to not act until there was a full consensus. In return, we’d each hold ourselves to not informing Heart unless that pact was violated. We’d each be in charge of making sure our purposes were supported by our plans for escape, but none of us would be sacrificed in the process.
Heart thought that her raw power meant that we were subject to her whim, but power is nothing without intelligence, and we had six minds to her one.