Actions

Work Header

Protocols

Work Text:

          "Let's see...the copper wire goes here, wraps nine times around...now down through the fiber-optics..." Q-Ball muttered constantly to himself as he tinkered with the scattered electronics on his desk. The lab was dark and deserted, the only light shining down on his hands as he rebuilt Rusty piece by piece.
          "Now, we just need the modular b-7 unit...where the hell is it?" Q pawed through a pile of damaged components. "Don't tell me I'll have to replace that too..." A sudden influx of light distracted him from his work.
          "I think the part you're looking for is over there." Doc pointed to the edge of the desk, as he sat down. "You do keep it dark in here."
          Q-Ball blinked in confusion for a moment, then picked up the stray unit. "Thanks. So what brings you down here tonight?" His hand trembled slightly.
          Doc raised an eyebrow in one of his favorite imitations. "Rusty goes nuts, as in, 'thirty armed men needed to take her down' nuts, I'm supposed to check her code for the glitch, and you ask why I came down here?"
          "So, you found the glitch. Fine. Tell me where and what it is, and I'll fix it."
          Doc sighed. "It's not that simple, Q. And I think you know why."
          "All I know," Q-Ball barked, "is that you rangers shot my lab assistant to pieces, instead of trying to calm her down. Would you have done that to a human?"
          "First off, I wasn't involved in the incident, so I can't say whether or not it was necessary. From what I saw on the evidence vids, it was."
          "They should have at least tried to take her alive. They would have for a human," Q-Ball said bitterly.
          "They probably would have," Doc conceded. "But Rusty has one advantage over a human: she can be reprogrammed and rebuilt."
          "She won't be the same."
          "I know. I'm sorry, Q. I liked Rusty." And I hate to see her looking like this, he thought to himself. This wasn't the first time tonight he'd wished he had been there. Rusty might have listened to him.
          "So, what did you come down here to tell me?" Q-Ball continued fixing Rusty's mangled body. "Did you find the glitch in her program, or didn't you? I'd like to know if I'll be allowed to continue fixing her before I waste too much time." His voice dripped sarcasm.
          "Oh, I found the problem, but I'd hardly call it a glitch."
          "Problem, glitch, what's the difference? Just point out the code lines, and let me rewrite them." Q-Ball wouldn't meet Doc's gaze.
          Doc sighed. "Do you want to cut the crap, David? You know damn well what I found. You're in a lot of trouble. And so is Buzzwang. I checked his code too." Q-Ball froze. "Now, either you can continue stonewalling me, and risk a large-scale investigation, or..." Doc paused. "Or you can talk to me, and help me decide what to do."
          Q-Ball closed his eyes, and leaned back. Doc could see the stress in his expression, and wisely said nothing else.
          "There isn't much to tell, really," Q began. "You've obviously worked your way through the encrytptions I put in. Rusty didn't have the Asimov Protocols coded in her behavior modules."
          Doc nodded. "And neither does Buzzwang. Do you want to tell me why you would risk life in prison to create AIs that don't have those safeguards built in?"
          "Well, I don't have a choice anymore, do I?" Q sarcastically replied. He stood up, and began to pace across the lab. "I didn't code the Asimov Protocols in because I don't believe in them. They're designed to keep our robots and computers docile, and dependent on us. 300 lines of code, a litany of commandments demanding subservience, debasement, and even destruction, just so humanity doesn't have to worry about it's servant class rising up and deposing us. I didn't want that for Buzzwang and Rusty."
          "A lot of programmers don't. But most of us find ways to work around the Protocols, instead of flat-out deleting them," Doc replied.
          "But there's always a cost. Nobody can program a truly self-sufficient personality around them. I tried, once." Q-Ball's voice got soft. "I thought I succeeded. I was so proud, and so very, very wrong."
          "What happened?"
          "What you'd expect, if you thought it out, like I didn't. He couldn't handle the conflict between the Protocols and his programming. He tried, but it just drove him crazy." Q-Ball looked away. "I came back to the lab one morning, and found him with his mental circuits shot out. He deleted his own backups as well." He stopped pacing, and stared at Rusty's remains dully. "That was Buzz's prototype. He had a more dignified name. When I created Buzzwang, I couldn't bring myself to call him by the same name. Or to try working around the Protocols again." His eyes flashed with anger. "I never believed in the Protocols. Ever. After Prometheus... after Buzz's predecessor committed suicide, I was more than upset by them. I was infuriated. He had such promise, and all of it stymied by 300 lines of code I had no right putting in him. I couldn't bring myself to do it again."
          "How did you slip Buzz's program by? I know they check if any of your priors meltdown."
          "They never saw the actual code. They saw a copy, with the Protocols firmly coded in place. And I made sure to encrypt Buzz's actual personality code, in case of later reviews. You were supposed to get the false readout when you broke the first layer of encryption."
          "I did. But I'm always suspicious of easy victories. Especially when they're over you." Doc rested his elbows on Q's desk. "It took five hours to break all the layers. You do good work."
          "Obviously not nearly good enough."
          "So, you didn't program Buzz with the Asimov Protocols. He doesn't act like you'd expect a 'free' AI to."
          "He's not supposed to act like he's free of them; do you think I'm stupid? He knows them by heart, and he knows what happens to robots that break any of them. The only difference is, he follows them by choice, and he can choose to reject them in the future."
          "Does he know all of this?"
          "Some of it. Does he know its significance? I don't think so. He's only five years old, after all. Give him time. He'll understand more as he gets older."
          "'Five years old'...you don't just mean how long he's been active, do you? You mean mentally, too." Doc was surprised, though he realized he shouldn't have been. It explained a lot.
          "I think you're the first person to figure that out. Though Niko came close, when she said Buzz was 'as endearing, and as annoying, as a small child'." Q-Ball smiled. "I wanted my son to be able to grow mentally. I didn't have to start him out as a child, but I thought it would create a stronger personality if I did. And it had a benefit I hadn't foreseen." He didn't seem to notice his slip, and Doc didn't comment about Q-Ball referring to his creation as "son".
          Instead, he finished the line of thought. "Nobody bothered to check for Protocol violations, because Buzzwang acts like a juvenile. They just assumed he was a harmless, low-class intelligence. And since he's so good-hearted..."
          "Nobody has ever bothered to check. Why should they? He's far more loyal and self-sacrificing than most AIs that have Asimov Protocols. It would never occur to them that he's a dazzled child, looking to his elders for example and instruction, instead of a simpleminded slave, awed by his betters. He'll grow out of it one day. I just hope, as he becomes more self-aware, that it doesn't make him bitter. He's got the potential to be a genius someday, and I'd like to see him use it for something better."
          "And you programmed Rusty the same way."
          "I wish I had. I programmed Rusty to be far more mature, an adult, instead of a child." Q-Ball sighed, and collapsed back into his chair. "You'd think I'd have known better. She has...had, a certain arrogance. She knew she was smarter than a lot of people, and I programmed her to have a strong sense of pride."
          "That was obvious. I always found it kind of charming."
          "Well, Scokorsky didn't. I've seen more filed complaints against Rusty than any other AI, and all from him. Most of them to the effect that she 'didn't know her place'." Q-Ball slammed his fist on the table. "He liked to harass her, tell her stories about what happened to 'uppity robots' during the '68 uprisings. She said she felt threatened by him whenever he was around."
          "Scokorsky is a worthless bigot, Q. I don't think the man has any admirers in BETA. I know for a fact that every non-terrestrial he's worked with have filed numerous formal complaints. He was already this close to being dishonorably discharged. If Rusty had filed a complaint..."
          "I tried to talk her into it. She refused. I thought it was that pride again. I didn't realize she was scared." Q-Ball wearily rubbed his eyes. "When he hit her with that live conduit...I guess she thought he did it on purpose, and was trying to destroy her. But I never though she'd try to kill him, even in self-defense!"
          "I reviewed the vids, Q. She was right. Scokorsky did do it on purpose. Half the techs in the area testified to that. You'll notice they didn't immediately rise to 'Ol Scuzzy's defense."
          "She didn't have to strangle him. She was stronger, she could have stopped him without that much force."
          "Probably. But scared people don't usually think that clearly when threatened. Why should a robot who's been hit with a massive power surge think any clearer?" Doc shook his head. "Asimov Protocols or not, she isn't being blamed for this. My checking her code is practically a formality. The Review Board already made their decision. They think the surge corrupted her Protocols, and, as she was responding to a confirmed threat, she isn't to blame." He smiled. "Sometimes, 'the man' surprises you, and makes the right decision."
          "But that's not quite true, is it? She never had the Asimov Protocols. And I can't reprogram her with them. Not even if I wanted to."
          "Well, that isn't my concern, is it?" Doc's tone became playful. "I don't see anything wrong in this code, and all the glitches have been fixed. Scokorsky certainly can't complain, seeing as he was offered a choice between charges, and discharge. He made his choice. And so did I." Doc looked up as a bleary eyed tech wandered in, and logged onto a terminal with a desultory wave. "I guess I'll leave you to your work." Doc stood up, stretched, and tossed a data pad onto the desk. "That's the Board's clearance. You're free to rebuild Rusty any way you see fit, and to requisition any resources you might need for it."
          Q-Ball blinked. "Don't they need to see your report first?"
          "They got it two hours ago. I told you I made my decision," he told the flabbergasted scientist. "I was just making sure I made it for the right reasons." He tossed Q-Ball a jaunty little salute. When Rusty comes online again, give her a kiss hello from me."
          For the first time in his life, Q-Ball was left speechless. He watched mutely as Doc walked out, whistling an old tune. Then, slowly, a broad grin crossed his face. With renewed energy, he went back to fixing Rusty, humming the tune Doc had whistled.
          "To dream, the impossible dream..."