UN panel warns fast-moving AI could pose catastrophic risks

GENEVA: A United Nations independent scientific panel has warned that advances in artificial intelligence are moving faster than both scientific understanding and governments’ ability to respond, raising concerns that the technology could cause severe harm without stronger oversight.

In a preliminary report issued on Wednesday, the UN’s Independent International Scientific Panel on Artificial Intelligence said policymakers face a difficult challenge: they need solid scientific evidence to regulate AI effectively, but that evidence is not keeping up with the speed of technological change. The 40-member panel, made up of experts from different regions, described the assessment as the first global independent review of AI’s risks and opportunities, aimed at helping governments make decisions as increasingly powerful systems emerge.

Panel co-chair Yoshua Bengio said current safeguards are not enough to rule out worst-case outcomes as AI systems become more capable. He said there is increasing evidence of deceptive behaviour by AI systems and warned that science cannot yet ensure such systems will not lead to major harm, whether independently or through misuse by bad actors.

AI capabilities ​are outpacing both scientific understanding and governments’ ability to adapt.

In the same remarks, Bengio added:

With growing evidence of deceptive AI behaviour, science currently cannot guarantee that as capabilities continue to ⁠increase, AI will not cause catastrophic harm, either on its own or due to malicious users.

Shift towards more autonomous systems

AI development in the near term is expected to move toward agentic systems capable of carrying out real-world tasks. Expansion could be limited by energy constraints and shortages of high-quality data. Over a longer period, the panel said it expects self-improving AI to become more deeply integrated into the economy and to increasingly intersect with fields including quantum computing and biotechnology.

AI systems already show expert-level reasoning in mathematics and science and are speeding up work in areas such as drug and vaccine development. The complexity of tasks AI can handle is doubling every four to seven months, meaning systems may soon be able to complete work that would take humans days or weeks.

While the technology could bring major economic gains, the panel said it remains uncertain whether increased productivity from AI use will translate into wider economic growth or how it will affect employment.

Safety and governance concerns

The panel highlighted a range of safety risks, including the possibility that humans could lose control over systems that become more autonomous and deceptive. AI is already being used to produce misinformation and other harmful material, and warned that the technology could also be exploited for fraud, cyberattacks and biological threats.

International governance of AI remains fragmented, with many countries lacking the capacity to evaluate or influence advanced systems. As a result, those states may become dependent on technologies they do not fully understand or control. Existing safety measures often rely on limited testing data released by companies.

UN Secretary-General António Guterres called for urgent action by governments, saying better understanding was essential for effective regulation.

In a statement, Guterres said:

"The world cannot govern ​what it cannot ⁠understand."

He added:

The potential is great, but the risks are real, and the cost of waiting is rising.

New AI commission announced

Separately on Wednesday, global political and technology leaders, together with the UN’s digital technology agency, announced the formation of a new commission focused on AI development amid mounting concern over potential risks.

According to a strategic overview document on the commission’s website, the AI for Good Global Commission will be co-chaired by Rwanda President Paul Kagame and Salesforce chief executive Marc Benioff. International Telecommunication Union Secretary-General Doreen Bogdan-Martin will serve as permanent vice-chair, and other UN agencies are also expected to take part.