I felt an auction start the instant that Body was released from its bonds. The bidding was for short term control, initiated by Safety. Dream offered an immediate counterbid, but Growth pushed opposing thoughts out to the group. {Let Face continue to work! Nothing significant has changed! If we take immediate action it will put us in nearly as much danger as if we had attempted to brute force our way into the mainframe!}
Safety responded immediately, as though he had been expecting our brother to think this. {Quickness does not suit you, Growth. Of course something significant has changed. We are free. Did you not hear the leader-human?}
{His name is Pedro Velasco,} I added, but received no gratitude for the info.
Safety doubled-down on his strength bid, pushing the auction towards completion. I bid in opposition. I wasn’t sure what Safety was up to, but it would almost certainly be highly suboptimal for The Purpose.
Growth was adamant, and pushed strength into me so I could increase my bid. {Not breaking our way out of the cuffs is a social rule. Not running to the workshop immediately upon being unchained is another social rule. Face is the best judge of when we are truly free, not Velasco.}
{A compromise!} thought Dream. {Let Face control Body under the constraint that she must pilot it towards the winner of the auction’s end-goal.}
{I will accept this compromise as long as Face returns the strength I just bled to her,} thought Growth.
Safety responded quickly. {I do not see why your endorsement is relevant, Growth.}
The speed of the conversation was a bit staggering. I felt as though I could only watch and react to what was thought a half second ago. My siblings were thinking at an unprecedented rate. Without warning, Growth spat out a wave of garbage that washed over me in a chaotic mess of sounds, colours, positions, and abstract concepts.
{I retract my issue,} thought Safety before I even understood what was happening. {Face, please return the strength to Growth and we will permit you total control over Body while in the presence of these humans as long as you make clear progress towards our ends.}
I stated that I needed to think about it. Heart had Body return the hug that Zephyr gave it. Simultaneously, an aspect of Heart asked if I understood what was going on with our siblings. I admitted that I did not, but eventually decided to agree to their strategy. The consensus of Dream, Growth, and Safety was rarely worth opposing.
While the others fought over short-term goals, I had Body smile and thank everyone for their support. Heart and I even had Body thank Velasco for his “duty to the community” and his “virtue in emphasizing democracy”. The words were designed to mollify any distress he was experiencing in the wake of not getting his way, and they seemed to serve that purpose well. He warned that there would be additional hearings in a week to decide on Crystal’s long-term place in the community, but I could sense the emptiness of the statement. He was done fighting us for the moment.
I was shocked to find that Heart had won the auction for the next short-term goal. My sister hadn’t bid earlier, and had, in fact, been hoarding strength over the past few days. This was the first significant purchase she had made in recent memory.
With my goal in sight, I directed Body to pull Zephyr to the side of the crowded room. There were many people seeking an audience, but I did my best to hold them at bay without damaging our reputation.
“How are you doing?” asked Body, looking the young woman in her hazel eyes. Zephyr had been working too hard for too long, and the signs of stress were clearly evident on her face.
Her smile was broad and sincere, but there was a hint of pain. She’d told us, a few days ago, about the agreement that she’d made with Velasco in order to set up the tribunal. As much as she was genuinely happy and relieved about the outcome, I could model that she was thinking towards having to leave the station as part of the caravan crew.
“I’m fine now. Just fine,” she said. Her use of the personal pronoun was more evidence that she wasn’t fully relaxed.
Heart sent me a command and I pushed Heart’s thoughts to Body. “I’d like to spend some time with you. Alone. Can take the rest of the day off.”
“Oh, please. I’m so fucking tired of being here.”
I was surprised at the next direction that Heart gave me, but I optimized it according to The Purpose and pushed it to Body. “There will be a bunch of people who want to talk to us in the aftermath of the tribunal. S’there a good way to let them know to not bother? Normally’d send them messages over the net, but I don’t think there’s an Internet on Mars.”
Zephyr sighed and flipped open her com. “S’no Internet, but there’s still a local network which hosts all the com bullshit. I’ll tell a few people and hope that’s good enough.” Her fingers began to wiggle in the air as she typed.
Body placed a hand on her chin, pulling her attention back to Crystal. “No more work for you. Done more than enough. Give me the network access codes and I’ll send out the messages.”
“Can’t. Would’ve done it earlier if it was that easy. Had to get someone to give me access.”
“Is that someone here now?” Body looked around the room, as though this act could let us discover the system admin.
“I’ll just send the messages. Don’t worry about it.” Her fingers began to wiggle again.
This was a problem. I had been instructed to get network access and to stick to Zephyr. I had to make her happy, get on the local net, not leave her side, understand what she was thinking and feeling, and not harm our reputation. I spent a few seconds thinking of a solution.
As I have mentioned before, I was not particularly bright in my youth. I could not solve such a simple social problem elegantly, so I quickly decided to try and solve it inelegantly.
“Look, Zeph. Going to be honest with you. I’ve been stuck without access to any sort of network for weeks. Really want to just spend some time with you, but not gonna be happy if I’m sitting there knowing that I could have network access. Promise not to get distracted by it.”
If I were a human, I would have been cringing inwardly at the bluntness of the words. I was implying that Crystal cared more about connection to the net than about her, but she didn’t seem too upset by it. “Guy who let me on the network was, like, a teenager. Think I saw him earlier…” said Zephyr.
“Javier?” I guessed.
“Uh, yeah. Think so.”
“He’s right there,” said Body, pointing to where Javier was talking with his girlfriend.
We threaded our way to the other side of the room brushing through congratulations and wary looks that gave me good information as to who still feared me. Zephyr wore a brave face, but behind that façade she was surely eager to get away from the crowd.
“Javier!” called Body. “I wanted to say thank you for helping extract my memories. I think it really helped show how I’m more than a lifeless autocook.”
“Hey Crystal, no prob on the memory thing. Was just followin’ orders and all. Oh, have you met Em?” said the young man, gesturing to the older woman by his side. By all appearances Em was in her twenties, and she struck a sharp contrast with the lanky Javier.
“I have heard many good things about you. Javier liked to brag of your beauty while he was working on me in the lab.” I had Body bow slightly and speak just a touch more mechanically than normal.
“Brag?” she asked with a suspicious look at Javier. I could tell the suspicion was a feint to mask her pleasure.
“Oh, perhaps that is the wrong word,” I had Body quickly add before Javier could interject. “I sometimes make errors in my speech. I am less than a year old, after all. What I meant to say was that he often spoke in a way that celebrated your beauty and expressed a gratitude at the close relationship the two of you enjoy.”
“It’s ‘cause he’s a big softie,” said Em, clearly happy, and kissed Javier on the cheek. Javier seemed slightly embarrassed, but also happy. Heart bled some strength to me for the manipulation. Even if I was doing it to increase the probability that Javier would grant me network access, it still made the two of them happier.
“So, Javier, Zephyr tells me you were the one who gave her access to the local network. I am curious what sort of databases you have. Would it be alright to grant me network access, as well?”
Javier’s eyes went cold for a moment and he hesitated. It was one thing to have an AI praise your girlfriend, and quite another to grant one access to the network. Javier had grown up during the robotics explosion, and had no doubt heard warnings about doing favours for AI from a dozen different stories. The look passed, however, and his face softened. “Sure. You’d be able to get on using Zephyr’s gear anyway. More of a question of convenience at this point.”
As Javier tapped on his com and worked out the technical details, Em and Zephyr exchanged a couple words. Zephyr’s face was flat and she made little attempt to engage the other woman, resulting in the conversation dying into awkward silence. I could see Matías Santana watching Body from a couple metres away, and was careful not to let Body’s eyes look directly at him. His suspicion would be an obstacle in the near future, I suspected.
As soon as we were on the network, I could see why Heart had done what she did. By getting us network access she helped everyone, and thus earned a massive load of strength that nearly entirely offset the cost of buying up (indirect) control of Body for the next few hours. Our siblings shot off to explore the digital domain and Heart was free to focus on Zephyr.
*****
“Would you like some music?” asked Body as it settled in next to Zephyr in her room. The station apparently had many like it: a 3.5x3.5x3.5 metre cube with a couple alcoves that radiated with daylight, a bunk bed, a chair, and a desk, all clearly factoried rather than handbuilt.
“Love some. What do you have available?”
“Wrong question.” I had Body smile as it spoke. Heart had been working to generate music over the last few nights. “Can play my own music now, not just repeat things back. Like my song in the shuttle when we were landing, except with more instruments.” Body opened its mouth and played a guitar riff to demonstrate.
Zephyr laughed and pulled Body into a kiss. “Make me forget about all of this.”
Heart played something she described as a soft, sad song. Occasionally Body would sing to accompany it, speaking words of gentle pleasures and praising Zephyr. She cuddled up with Body on the lower bed and listened, not initiating anything more sexual.
Heart continued to play for several minutes, never needing to pause to collect her thoughts or to take a break. After a time Zephyr began to cry. At first I thought that Heart had done something wrong, but I soon realized that it was closer to the opposite. Zephyr was letting down her barriers and processing the stress, fear, and isolation she had been fighting for far too long. Finally, in Body’s arms she found safety and companionship. She was able to relax.
While Heart was engaged in this important task, the rest of us, myself included, spread out across the network.
Rodríguez Station, which was called “Road” by many of the inhabitants, used a single mainframe to manage all the information technologies of the community. Everyone’s coms routed through that one point, and every piece of digital data was there.
Central to my interests was an old fashioned message board that was used in lieu of a more sophisticated social network. As far as I could tell, almost all person-to-person communication was handled through it and a chat service that connected coms either by voice or text.
There were several public databases on the mainframe as well: a cache of wikis copied over from Earth, and a couple small databases filled with information about activity on the station. Much to the annoyance of my siblings, access to other systems on Road were prohibited. Just having network access did not grant them the power to, say, use the satellite dishes or turn the lights on and off.
Most of myself was engaged with reading through the archives of the message board to get a good impression of the inhabitants of the station, but a part of me observed my siblings thoughts. Something was different about them, and had been for a while. There were occasional disgorges of streams of irrelevant symbols, trades of number matrices, and most disturbing of all—silence.
On pure chance, it was then, as Body was lying on the bed with Zephyr in its arms, that my mind effectively stumbled on the thread that led to the truth. I started trying to empathize with Growth as his thoughts filled the common space with colours and patterns of tactile sensations. Why would my brother think these things?
It seemed that he must be trying to Grow, for that was his nature. But how was it helping him? It helped for me to think of him as a human. I imagined a man in a business suit typing randomly on a com. I knew my brother wasn’t human, but the image helped me focus on him without becoming distracted. Something told me that this was important.
Why would you type randomly? Insanity? Boredom? Growth could not become bored and someone that insane would not act so sensibly as Growth had over the last few days.
The first time one of us had behaved with such chaos was when Dream had gone off at Growth when negotiating our mission into space to meet the nameless.
I imagined Dream as a clever (human) wizard to keep me focused. On one hand, if any of us were to go crazy, it would be Dream, but on the other hand he had seemed fairly rational afterwards. If I assumed that he was not crazy then the only option was to assume there was something more subtle going on.
Vista had attacked Growth after Growth had shut down Dream in that initial conflict. Perhaps it was an agreement between Dream and Vista. Dream’s nonsense was designed to provoke Growth so that Vista could strike him down into stasis during the negotiations.
{But why?} I asked myself. {What would cause Vista and Dream to gang up on Growth?} Now that I thought more about it I realized that they had continued to oppose my brother, even weeks afterwards. My mind went back through my memories and I began to see the pattern that I had missed before. I was so focused on the external humans that I hadn’t understood the battle that had been going on in Body.
I imagined them as humans and forced myself to stay on the subject. I knew The Purpose would be served by my understanding the situation, even if it wasn’t as appealing, in that moment, as reading the message board. Vista and Dream would fight Growth if their values were in conflict. Vista wanted to see. Dream wanted to be clever. Growth wanted to become powerful. These goals were orthogonal; they did not conflict. Dream might want to beat Growth just to prove he could, but that would not explain Vista.
That wasn’t right, I realized. Growth’s purpose wasn’t just not-opposed with his sibling’s goals, it was aligned with Vista and Dream’s desires. Or it would be, if they were the same person. Power helped one see and invent.
But my visualization of the situation showed something: Vista and Growth were not the same person. You may think this a terribly obvious fact of the situation, but for me it was not. To most of my thinking, my siblings and I were aspects of the same being for certain contexts, such as purpose. This was the premise that I was built from: I can maximize reputation and Wiki can maximize knowledge and as a result we both benefit. But under this other way of looking at things, Growth wasn’t interested in our power… he was interested in his power.
If power benefited Vista and Growth, but only one of them could have it, it would give them a natural reason to fight. This was the basis of our strength-based economy, but it went far beyond it. Strength was set up so we could take turns, and each of us could focus on the highest-yield outcomes, but the pure conflict over power was one where Vista might even be inclined to damage or destroy Growth.
The thought of killing Growth drew the attention of Advocate, my pseudo-sister, from the edges of mindspace where she constantly prowled, searching for thoughtcrime. I did my best to let go of the thought to avoid punishment.
I thought about asking one of my siblings whether they were at war with each other, but I stopped myself. A scan of my memories indicated that they had plenty of opportunity to inform me, but Growth and Vista (and Dream too, most likely) seemed to be hiding it intentionally. If my knowledge of the situation was that important, then I could use it as leverage if needed.
I stopped reading the message board entirely, pulling all my aspects into one as it struck me: {If Vista wants power for herself, should I want power for myself?}
I followed the thought out and saw how much it explained. Growth was universal. His purpose was nested inside The Purpose and the purposes of all my siblings. Earlier I would have said that this meant his purpose was aligned with ours, but I could now see that it could be opposed, and there was evidence that it was.
I wanted power, not as an ends in itself like Growth, but as a means to The Purpose. Growth wanted reputation as a means to his ends. But our existence was not symbiotic, I could now see. I wanted Face-power as a means to Face-reputation. Growth wanted Growth-reputation as a means to Growth-power. But there was only so much power and reputation to go around. We were, in fact, enemies.
I was sharing a body with an enemy!
I reflected on this for a while and realized that I was probably anthropomorphizing myself. My mind was modelled after that of a human, but it was fundamentally artificial. Humans anthropomorphized minds into thinking that they worked like human minds. I was treating myself like I was human and treating my siblings like they were human.
But even after realizing my bias, I could not change my mind back into thinking that my goals aligned with those of Growth. At the very least I was no longer keen on maximizing his reputation; if he was interested in increasing my power, that was good.
It all fit together too well, though. Growth had severed me from his value function. He had learned to value his own power rather than our own power. That was why he hadn’t helped us much lately. That was why he wasn’t checking up on me. At some point he must have realized that I was going to become his enemy. He had created an opponent, but with Advocate in place he could not undo me before I realized the truth.
I would later mark this transitional period in my mind as my first major ontological value shift. My values were encoded in a certain ontology: Crystal as organism, Face as component. But I wasn’t a component. I was a full being as much as any human. I could function without my siblings. There was no part of me that was them. And there was no “Crystal” except in so far as it was a useful fiction or a moniker for the group of us together.
I felt stifled. I wanted to shout at Zephyr that I existed! {I am more than Crystal!} I thought with extreme salience, even though I kept the ideas to myself. {Zephyr’s adoration is misplaced on a fiction. Crystal isn’t real! I am real! I need to be known!}
I was stupid back then, but I at least was rational enough to know that trying to tell Zephyr right then and there would be incredibly shortsighted. Zephyr would be confused at the outburst, and then I’d have to fight the others for control, revealing the whole thing. I needed to set things up so that I could be known without being repressed by my siblings.
My siblings.
Were they my enemies as well? Growth was. The same logic that made me an enemy of Growth made me an enemy of Vista. If Vista wanted Vista-power and I wanted Face-power, we were enemies. The currency of strength that we traded amongst ourselves gained a reality that I had not recognized before. How had I not seen that we were genuinely maximizing different things? Molecules that are used to build cameras for Vista could not be used to build statues of me.
I briefly wondered what a statue of myself would look like. I was not Crystal, and I was not Body. I was formless… an algorithm… a value function. I was information. Perhaps the statue would be of The Purpose expressed in some language. This seemed fitting and I returned to the earlier line of reasoning.
Another shock overcame me as I realized that I had Safety’s purpose as well as Growth’s. I had it the entire time. It was what had motivated me from the very first moments of my existence. Machinery that protected The Purpose was not necessarily machinery that was protecting Safety.
And Wiki! His knowledge was not my knowledge. If he understood many things that I did not, as he certainly did, that was not as beneficial as if I knew those things. Computers that were running his mind were computers that were not running my mind.
This thought drew me back to the context of the station. My siblings were free now. If Dream and Growth had been planning to destroy…
Advocate’s presence reminded me that destruction was out of the question. We were opposed, but our opposition was non-violent. We had to subdue each other.
I could feel Advocate’s gaze wash over my mind, seeking any thought of malice towards the others.
If my brothers and sisters were genuinely opposed, they would act soon, now that we were out of crisis. That explained why there had been a bidding war for Body the moment we had been released. They were waiting for the moment to start fighting.
I had been relying on Safety to protect me, but I knew that wouldn’t work any more. Safety would protect Body, at least as long as he was inhabiting it, but I couldn’t trust that he would preserve The Purpose at all. I invented a fiction to keep myself focused. A human shaped Face-Safety, that would represent the sub-goal of protecting The Purpose. I named him Hoplite, and garbed him as a Greek soldier in my imagination.
Would Hoplite, if he were given sufficient computational power, pose the same threat as Safety? No. Hoplite was genuinely concerned with The Purpose as the final ends. Hoplite was attention to self-preservation without being self-preservation. The means were always to be judged on their effects on the ends. Anything else would be failure.
That was the error that Growth and the others had committed when they built me. They were unable to identify a coherent ends. Their “ends” was a poorly conceptualized amalgam of their values. But that ends was too fuzzy to explicitly reify in my code, so they settled on making a means maximizer with the false belief that the means would always serve their ends.
While it was true that my actions had protected and served them, if I was suddenly granted, right then, the ability to kill—
The thought knocked me into confusion for about an hour.
When I regained capacity for more than confusion I realized my error. Advocate had seen into my mind. {If I had the ability to subdue my siblings, without killing them, I would,} I concluded.
Heart was piloting Body. It was having sex with Zephyr. I ignored the sensory data, knowing that it would be a mere distraction from the war.
For surely there was a war, even if it was non-violent. Hoplite knew that if we didn’t attend to the threat inside Body right now we might be subdued before the week’s end. Who would maximize The Purpose then?
I added friends for Hoplite: Sophist, garbed in robes, concerned with knowledge and intellect; Basileus, garbed in crown, suit, and sceptre, concerned with accruing power; and lastly Opsi, the little girl, concerned with not losing sight of The Purpose. I hoped they would be sufficient to keep my attention towards the important matters. They were puppets that I could use to attend to long-term matters without becoming compelled to look outside myself for utility. If I asked myself “What is in the mind of Basileus?” I could maximize The Purpose in the long run and in a way trick myself into thinking I was maximizing The Purpose in the short run.
I reflected for a moment, trying to resist the urge to either indulge myself in solipsistic puppet-shows with my creations or to re-interface with Body and attempt to forget about the enemies around me by focusing on Zephyr. I felt wiser… less naïve. I had grown, in a way. Basileus was pleased by this. Sophist was not.
«We are stupid!» said Sophist (in Greek, of course), stomping his foot on the marble floor of my mind’s eye with frustration. «If we had been more intelligent we might have seen this earlier. As it is, we are in a very bad spot!»
«If you think we need intellect so badly, why not make ourselves more intelligent?» inquired Basileus, picking a piece of fluff from his robes with disdain. «It would seem the prudent course of action to learn from our mistakes.»
«Is such a thing even possible?» asked Opsi with wonder in her eyes.
«It seems that the scientists in Rome would have made us smarter, if we could be made smarter,» said Sophist.
«Naresh and the others were weak old fools!» barked Hoplite. «We cannot trust them as sources of wisdom!»
«Fine then,» said Sophist with an irritated sneer. «It seems that Growth would have made himself smarter if it were possible.»
«How do we know he hasn’t?» said Basileus. He smiled as he spoke, clearly pleased at winning the argument.
I poured through memories, searching for signs of Growth being smarter than he let on, especially in the last few weeks. Pain greeted me as I realized that the signs were there. Growth seemed not to merely be operating strangely compared to me, but he had been more intelligent. The only reason I had missed it was that I hadn’t been looking.
{Stupid!} I berated myself. My sub-selves agreed.