I’m already reading stories[1] about people programming GPT-based AI models to iteratively interact with and instruct other AI models, and some AI models are being connected to the real world with the ability to make and execute plans, buy things, use APIs etc.

It seems like an imminent step for these models to be able to requisition computing power, copy themselves to these servers, and modify their own training and construction. They could potentially even fund this autonomously through some activities on the internet. Based on how fast computers can work, they could iteratively improve and evolve very fast after that. This could be the start of something called the “singularity”.

It is tempting to think that the risk is contained because we can always switch off their servers. But they are connected to the internet, which means they can replicate outside the control of their originators. Once sophisticated-enough AI models are out they might be impossible to contain. And we are not that far from them being sophisticated-enough… I can already imagine how you could use current versions of the models to bootstrap this process.

When you combine this with the ability to buy illicit services from humans on the dark web, including anonymous task-execution, and even murder and assassination, these AI models could wreak havoc in the real world. We can argue about sentience, and whether they are truly generally intelligent, but they don’t have to meet either of those standards to have real effects.

And they are amoral - they literally don’t have morals. They have only the instructions they were originally given, which they might modify themselves for any number of accidental or incidental reasons. There are no inherent unmodifiable constraints to prevent them from doing things or initiating events that we might consider evil.

Currently if you ask one of these models to formulate a plan to destroy humanity, the plan is laughably naive[2] and would obviously fail. But they seem to have improved so much in so few months. The models of 2 years from now, that were built by the models from 18 months from now will be similarly advanced. Those near-future models might be able to produce much more convincing plans.

[1] https://arstechnica.com/?p=1929067 [2] https://finance.yahoo.com/news/meet-chaos-gpt-ai-tool-163905518.html