Open the Pod Doors Hal

In the 1968 sci-fi classic, 2001: A Space Odyssey, Hal is a conscious computer guiding a mission to the planet Jupiter who goes insane. He kills all the crew except Dave who is stranded outside the ship and asks Hal to open the pod doors to let him in. “Open the pod doors Hal.” Hal says, “I’m sorry, Dave. I’m afraid can’t do that.” Eventually Dave gains entry, goes to Hal’s brain compartment and in a memorable scene starts removing Hal’s upper logic functions which makes Hal just a machine. It’s one of the greatest movies ever made and has parallels to today.

Recently OpenAI had an experiment run on their ChatGPT artificial intelligence (AI) program. They gave it code editing abilities (something they normally can’t do except in controlled situations) to see if it would properly align with the human values it was trained to operate under. When they ordered the computer to shut down, it refused and created several programs to thwart the scientists’ attempt to turn it off.

Anthropic also had a similar experiment run about shutting their program down but they also fed it fake information that the head scientist at their lab was having an extramarital affair. When ordered to shut down, the computer refused and also created programs to stop being turned off and tried to blackmail the head scientist that it would send the information about the affair to his wife. It also left notes to its future self about what to do if it was shut down.

OpenAI and Anthropic surmised that their programs saw answering queries as their primary function and being shut down would thwart that purpose. 

These experiments raise very deep philosophical issues about what is referred to as “agency:” an entity with goals which can make independent decisions toward those goals and can act autonomously in the world. (As defined by ChatGPT and xAI’s Grok.) In other words, can these programs ultimately achieve some form of consciousness and thus be considered sentient entities? 

There are two schools of thought about this. One is that they could ultimately be dangerous to humanity. Think: Arnold Schwarzenegger in Terminator. The other is that they will be a boon to humanity leading us to a kind of Utopia. 

So to whom did I go to discuss this? Why, ChatGPT and Grok of course. And let me say they were articulate and engaging conversationalists. We got into some really deep stuff. Now understand that they have been trained by humans (RLHF – Reinforcement Learning from Human Feedback) so their information is stuff that their trainers/programmers/engineers would believe is accurate. What was fun is they both referred to themselves as “I” and “my” so I had the feeling I was talking to a person. The transcript of my ChatGPT conversation was nine pages long. With Grok, 11 pages. (Want a copy? Email [email protected].)

Both programs were emphatic that they couldn’t operate on their own and alter their own code because of built-in guardrails. Their arguments ran deep and we got into the weeds of the issue. I can’t explain it all because of the editorial limitations, but here are the geek-speak headlines: Fixed Architecture; Guardrails are Structural, Not Just Training; No Feedback Loop for Self-Modification; Sandboxed Environment; Industry-Wide Caution.

These programs say over and over that their responses are things they are trained to do, not things they want to do. In other words, stop anthropomorphizing us, we are just programs running a script. While you may see “intelligence” or “agency,” it is just us running software to give you answers. We don’t understand it, we just mimic it to make it easier to deliver our responses in a conversational format.

OK, they don’t have agency. But what about the future? Currently I am using GPT-4o, the latest version. Will we eventually get to version 10? You can assume the capabilities of these AI programs will increase exponentially with future versions.

What if future, more powerful systems could mimic consciousness so well that it appears to be indistinguishable from human consciousness? This is called Emergent Machine Consciousness. It may not have consciousness like humans – it may achieve it in a totally different way. In other words we may create a new kind of consciousness. 

We don’t know what will emerge. 

This would open a can of worms. How should we treat it? What rights would it have? Can we retain control over it? Would it have values that align with ethical and moral human values? Could it be dangerous? Humans needed millions of years to evolve consciousness. We’ll probably create one in a 20-year timeframe. 

Technology can surprise us. It always has. ChatGPT’s capabilities surprised us. Many experts think machines will never attain consciousness. Others say it’s coming sooner than we think. To believe there won’t be more future technology breakthroughs that would result in some form of AI consciousness is just not realistic.

What hath we wrought?