A leading British politician has urged the United Nations to intervene in “the unsafe development of super intelligence” after an Anthropic team leader posted: “We really do earnestly believe AI could kill all humans!”
Whether “marketing hype” or signposting of a genuine extinction threat, “governments must step in,” Labour MP Darren Jones wrote in an open letter also addressed to the UK prime minister and international Organization for Economic Cooperation and Development.
Jones is the latest lawmaker scrambling to find ways to protect the British public from AI threats to safety. Just one day earlier, Labour MP Alex Sobel had introduced a parliamentary bill proposing to ban the “development, deployment, and operation of artificial superintelligence”—systems which can fully replace and outmatch humans at any task. The legislation would also give the government powers to monitor and control the development of the technology at different points in the supply chain, including at chip level.
“We are not alone in this effort,” said Sobel while introducing the bill, pointing to the US government’s temporary export ban on Anthropic’s Fable 5 and Mythos 5 models in June. “If the Trump Administration, who are perceived as historically against AI regulation, can contemplate drawing a line on models that are too powerful and dangerous, it would be strange for us not to ask what our response should be,” he argued.
MPs from across political parties put their names on the bill, including Labour MP Jess Asato, who is suing Elon Musk’s xAI after its Grok chatbot was allegedly used to generate fake sexualized images of her. Lawmakers have become increasingly worried about the threat of AI to national security after a string of incidents of AI agents going rogue in recent months. They want to introduce guardrails against worst-case scenarios, such as these systems seizing control of critical infrastructure and running sophisticated influence operations.
The Hugging Face hack by OpenAI’s models that escaped containment in July “is why it’s even more important that we move quickly to regulate and legislate on superintelligent AI,” Sobel tells WIRED, “because AI is developing at pace, and I’m afraid to say those developing it don’t know exactly what’s in it.”
In an unusual move, the parliamentary bill was drafted by the nonprofit ControlAI, which is backed by Skype founding engineer and former DeepMind investor director Jaan Tallinn. He had previously been working at AI safety startup Conjecture but tells WIRED he wanted to make the public and policymakers aware of the “open secret” that “developing superintelligence could cause human extinction.”
Conjecture founder Connor Leahy now leads ControlAI’s US arm, which consulted on Bernie Sanders and Greg Casar’s bill to ban superintelligence development, in addition to having briefed nearly 200 congressional offices and more than 20 members of the Senate and House of Representatives on the extinction risk of superintelligence.
Sobel will present his bill in full in November. While he accepts it’s rare for the UK government to accept private legislation like this, Sobel hopes it may lead the government to work with other countries on an international ban.
Sobel has previously tried in vain to introduce a “kill switch” into the Cyber Security and Resilience Bill, which would allow the government to pull the plug on any data center hosting an AI agent that presented a threat. The mechanism, which was also proposed by ControlAI, is similar to a law pushed for by both Republican and Democratic lawmakers in the US. AI minister Kanishka Narayan, then working under former prime minister Keir Starmer, shot it down, pointing out that the bill already allows the government powers to intervene if compromised network and information systems cause a national security risk. That could include requiring an entity to cease using and isolate an AI model, he noted.