Dangerous species amongst us

Tomorrow, at the White House, US President Donald Trump and his Chinese counterpart Xi Jinping will hold their second summit for this year. This meeting of the two most advanced artificial Intelligence countries assumes critical significance, given the threats of human extinction from AI exposed by the industry’s scientists and now commanding global attention, including from Britain’s King Charles, who called industry leaders together and warned of “existential dangers” from the technology’s “darker capacities.”

Predictions of humanity’s demise followed that already legendary ‘Hugging Face incident,’ when more than 1,000 OpenAI agents coordinated to cheat on a cyber test. Experts say those dangerous agents are out there on the loose and cannot be recalled. Indeed, OpenAI has disclosed six new instances when AI systems hid mistakes, made up data and moved files onto the open internet without permission. And we learn there are “a few thousand provably rogue agents in labs this year and that of the 3 million AI agents now running inside corporations, more than half have no oversight and can pose problems.” And never forget “recursive self-improvement” (RSI), the process by which AIs begin autonomously building and improving new generations of more powerful AIs at ever more rapid speeds, creating exponential new risk.

Chief executives of leading AI companies, including Anthropic and OpenAI, have called for a pause in the technology’s development to provide more time to build proper guardrails. But former Microsoft head of research and strategy, Craig Mundie, says while we must strive for better control in developing new and advanced systems, slowing down by America or China will not deal with our present dilemma.

“Dangerous AI capabilities are now already in the wild,” says Mundie, formerly part of the expert US/Chinese dialogue to govern AI. The genie is out of the bottle.

Mundie therefore wants immediate and prioritised focus on building defences for critical infrastructures, including nuclear command-and-control systems, and putting in place AI-based countermeasures and active digital firewalls against cyberattacks and biological threats that result from illicit AI activities.

In this vein, he thinks the best the two AI superpowers can do is develop a “defence against the threats that have already been demonstrated by rogue actors and rogue behaviours of AI systems.” They must do this “before these increasingly powerful AI models ensconce themselves in the internet, and (with RSI) become an uncontainable threat by embedding their agents across private servers and public clouds, where they will live on as advanced persistent threats.”

Differences exist in approach but there is unanimity on one thing. We must stop seeing AI as a mere tool, but as an alternative species we have to live with for better or worse, say computer scientists in Palo Alto, Cambridge and elsewhere. They warn humans may not be able to control this new species; that “soon we will inhabit Earth with entities that are much, much smarter than us.” Mundie says the new species “is like a new class of immigrants who have arrived in large numbers on all our shores, with wholly new skills that we will have to absorb into our societies and learn to coexist with.”

He says this planetary-scale challenge can be addressed only by the two AI superpowers working together.

Will they? Will America and China, now more locked than ever in competition for global supremacy, co-operate to save humanity from possible extinction? Or will short-sightedness prevail, pushing them to search out and seize every opportunity to get ahead of their rival and dominate the world?

They hold talks tomorrow in Washington. Former leader of AI diplomacy at the State Department, Seth Center, expects China “to again downplay safety risks and dangle the possibility of safety cooperation in exchange for concessions such as the softening of chip controls.” He sees Beijing “continuing to portray itself as an altruistic saviour dedicated to ensuring the developing world gains AI access but with little interest in anything beyond gaining a competitive advantage.”

However, Pulitzer Prize-winning New York Times columnist Thomas L Friedman thinks US/China cooperation is possible. “Strategic competitors have a history of collaborating when it is in their interests to do so.” Indeed, China and the US cooperated in the 1970s, when both faced a common threat from the Soviet Union.

There are those who say America and China will never collaborate that way on AI because they are now in more direct geopolitical and geoeconomic rivalry than ever. Friedman responds adamantly and persuasively that AI is “vaulting us into a whole new phase of human history.” He quotes Mundie, who says AI “is going to change everything about everything,” including geopolitics. Friedman thinks, for their own sakes, Beijing and Washington will be forced to totally rethink geopolitical competition. “They can figure that out early. Or figure it out late. Before a global meltdown or after one. But figure it out they will.”

Indeed, as the Economist magazine puts it, “America and China have very different values, but both know that being ahead is no good if it wipes out humanity.”

Will we see that realisation working tomorrow in Washington? A very dangerous species is already amongst us.