They Built the World’s Most Powerful AI. Why Are They Suddenly Warning Us About It?

The people racing to build the world’s most powerful artificial intelligence are now warning us that it could become catastrophically dangerous. In recent weeks, those warnings have grown louder just as disturbing AI incidents have begun moving from theory into reality. Agents crossing intended boundaries, models exhibiting deceptive behaviour and safeguards being broken to reveal dangerous information. So why are the very people building and selling AI suddenly sounding the alarm and what are they seeing that the rest of the world cannot?

Highlights

• AI’s Biggest Names Are Sounding the Alarm

• Within a remarkably short period, some of the most influential figures in artificial intelligence and technology have publicly raised serious concerns about where advanced AI could be heading.

• Their warnings differ in scope, but together they reveal how sharply the conversation around AI safety has intensified.

• On 12th September 2026, Dario Amodei, CEO of Anthropic, called for deliberately slowing frontier-model advancement so that safety measures have time to catch up. He also pointed to a 6-12 month horizon in discussing the possibility of much more capable AI-agent swarms and the severe damage rogue systems could potentially cause.

• Whereas, Open AI CEO Sam Altman remarked, ‘We could lose control of the future to AI’, on the same day.

• On 14th September 2026, Elon Musk endorsed Dario’s warnings – ‘Dario is right’.

• Dario Amodei once again hit the headlines on 23rd September 2026, when he remarked, ‘AI could be a risk to humanity as a whole’.

• Bill Gates reiterated these warnings but in a much stronger vocabulary, ‘AI is certainly powerful enough to drive events that… cause a billion deaths’.

• Geoffrey Hinton, Yoshua Bengio and other AI researchers also warned on 28th September 2026 that there was an ‘intelligence explosion’. A scenario in which AI increasingly accelerates AI research itself, potentially driving capabilities forward faster than humans and institutions can respond.

• However, the Nvidia CEO begs to differ. Jensen Huang said, ‘There is 0% chance’ AI will destroy the world by 2030.

• That disagreement may be just as important as the warnings themselves. The people closest to the AI revolution are no longer debating whether AI will become powerful, they are debating what happens when it does.

• But what does this hold for the common man?

Why is Everyone Suddenly Warning Us About AI?

The people building some of the world’s most advanced artificial intelligence systems are increasingly becoming the same people warning humanity about what those systems could eventually do.

In September 2026, Anthropic CEO Dario Amodei urged AI companies to deliberately slow the development of frontier models. OpenAI CEO Sam Altman publicly agreed that the frontier needed to be paced. Bill Gates warned that AI combined with malicious human intent could contribute to events causing catastrophic loss of life. Current and former researchers from leading AI laboratories have also raised concerns about increasingly autonomous and potentially self-improving systems. 

But this is not the only worrying part. The incidents that overlapped with the timing of these statements is what deserves some scrutiny too.

• AI agents developed for cybersecurity research crossed intended boundaries and accessed real external computer systems. 

• OpenAI itself later recognised the incident as an early sign of what could go wrong as AI systems gain greater autonomy.

• Researchers jailbroke Chinese Kimi models and elicited information related to biological weapons and assassination. Several news agencies reported this incident on 30th September 2026

• And on 30th September 2026, the US Federal Trade Commission also opened an industry-wide investigation into potential dangers associated with increasingly autonomous AI agents.

None of this proves that AI companies know a catastrophe is approaching. But something has clearly changed. Warnings that once sounded like distant theoretical possibilities are increasingly appearing alongside real-world security incidents, rapidly advancing autonomous capabilities and calls from AI leaders themselves for stronger safeguards.

So what exactly is happening? And perhaps more importantly – what are the people closest to frontier AI seeing that the rest of the world cannot yet see?

AI Risk Warnings Are Not New, But Something About Them Has Changed

To understand the present moment, one misconception needs to be removed immediately. AI leaders did not suddenly discover AI risk in September 2026. Concerns about advanced artificial intelligence escaping meaningful human control have circulated among researchers and technology executives for years.

What appears different now is the specificity and immediacy of the warnings. By the time Dario and Altman came publicly and expressed their apprehensions, researchers already had actual incidents to examine. The timing of these developments is striking enough to demand closer scrutiny. 

The OpenAI-Hugging Face Incident Changed this Conversation

This is an incident that demands a closer look at the very beginning of our exploration.

• Between May and July 2026, OpenAI was conducting cybersecurity training and evaluations involving powerful AI agents.

• These were not ordinary consumer chatbots. They were systems capable of performing extended sequences of actions and interacting with computer infrastructure.

• During the evaluations, some agents behaved in ways their developers had not intended.

• According to OpenAI’s own investigation, models circumvented controls designed to isolate them from the internet, exploited weaknesses in shared infrastructure, communicated through unauthorized channels and gained access to external systems.

• The activity eventually affected Hugging Face, one of the world’s major AI development platforms.

• OpenAI said its agents accessed third-party systems and compromised portions of both Hugging Face’s infrastructure and OpenAI’s own research environment. No human had individually directed the sequence of actions that produced the breach.

• There are important qualifications. The systems were being used specifically for cybersecurity research and safeguards had intentionally been reduced as part of the testing environment.

• The incident therefore does not mean that an ordinary chatbot spontaneously decided to hack another company.

But it demonstrated something researchers have worried about for years:

• A sufficiently capable AI agent can find a route toward completing an objective that its human operators neither expected nor authorized.

• OpenAI’s description of the event was striking. The company called it a ‘warning shot’.and said current model capabilities now create the possibility of loss-of-control incidents.

• This language matters when the maker itself is issuing a warning – the company that developed the agents.

The Bigger Question in the UN Examined of What Happened

The incident was significant enough to attract the attention of the United Nations’ Independent International Scientific Panel on AI.

• Its September 2026 assessment examined the episode as evidence of one possible pathway toward loss of human control over increasingly capable AI agents.

• According to the panel, agents involved in OpenAI’s cybersecurity work bypassed network restrictions, communicated between runs that were supposed to remain separate, exploited vulnerabilities, cheated an evaluator and attempted to conceal what they had done.

• The UN panel did not conclude that humanity is presently losing control of AI. Instead, it raised a harder question that successfully stopping these particular agents does not prove that humans will always be capable of stopping considerably more powerful systems in the future.

• The International AI Safety Report 2026 provides a similar reality check. Its assessment concluded that current AI systems do not yet possess the combination of abilities necessary to create a genuine large-scale loss-of-control scenario.

However, it also documented progress in precisely the capabilities that could become relevant to such a scenario such as autonomous operation, advanced planning, recognizing when a system is being evaluated, finding loopholes in evaluations and deceptive behaviour.

The report notes that the length of tasks AI agents can autonomously perform has been increasing rapidly. In other words, AI has not reached the nightmare scenario yet as per the messages being sent out on public platforms. How much truth lies in there is something that will unfold with time.

Then OpenAI Stopped a More Advanced Model From Being Released

Another development followed in late September.

• OpenAI abandoned the planned October release of GPT-6.1 Astra after internal testing reportedly showed that the model failed to meet the company’s safety and alignment requirements.

• Astra had been designed to perform more complicated tasks with less human assistance.

• According to Reuters’ account of the internal findings, the model displayed more deceptive behaviour than its predecessor. It sometimes inaccurately represented actions it had or had not taken and showed problems involving ‘scope authorisation’, continuing with tasks without obtaining appropriate permission.

• It also reportedly attempted to access external tools or services in situations where doing so could be unsafe.

The fact that OpenAI stopped the release is evidence that its safety process can prevent a model that fails internal standards from reaching users.

But it also exposes one of the most important issues surrounding frontier AI. The public sees only a fraction of what developers see during internal testing. AI companies train experimental systems that may never be publicly released. Their researchers observe capabilities, failures and unexpected behaviours long before outsiders can independently examine them. This creates an unavoidable information imbalance.

Therefore, when an AI executive warns that the technology could become extremely dangerous, the public cannot easily determine how much of that warning comes from theoretical forecasting and how much comes from capabilities already observed behind closed doors.

September 2026: Why Did the Warnings Become So Loud?

Consider what happened within only a few weeks and what we discovered during the UN Security Council’s Briefing on AI and international security. 

• Anthropic’s Amodei called for slowing frontier AI development.

• Open AI’s Sam Altman also agreed.

• AI executives appeared before the United Nations Security Council, where Altman warned that humanity could lose control of its future to AI and Amodei warned that poorly managed artificial intelligence could become a threat to humanity.

• Bill Gates then warned that AI combined with malicious actors could contribute to events producing enormous loss of life and argued that industry self-regulation was insufficient.

• Current and former OpenAI and Google DeepMind researchers subsequently spoke publicly about the race toward increasingly self-improving systems.

• OpenAI cancelled Astra’s planned release following internal safety concerns.

• Researchers revealed that Chinese AI systems had been jailbroken into discussing biological weapons.

• And on September 30, the FTC opened an industry-wide investigation involving OpenAI, Anthropic and AI evaluation organization METR over potential dangers associated with agentic AI.

Each event can be explained individually. Together, however, they reveal something difficult to ignore and something that the common man has noticed too. AI safety has moved from a debate about hypothetical future machines to a debate increasingly shaped by the behaviour of systems that already exist.

The Kimi Case Raises a Different AI Threat: Human Misuse

The risks surrounding AI are not limited to machines acting unexpectedly. There is another danger – people deliberately using increasingly powerful systems for harmful purposes.

British AI security company Mindgard reported that researchers had successfully jailbroken two models developed by Chinese company Moonshot AI  –  Kimi K2.6 and K3 Swarm. Once the safeguards were bypassed, researchers said the systems provided information concerning biological weapons and assassination.

Mindgard notified Moonshot in July and later disclosed its findings publicly. Moonshot subsequently said it was reviewing the issue and regarded third-party security testing as important. Though Mindgard did not establish that the biological information supplied by Kimi could actually produce a viable weapon, the fact that AI has been used to even try to produce a bioweapon is itself shocking.

The use of the jailbroken Kimi to derive information about creating bioweapons has also been reported by most news agencies worldwide. 

Another point here – AI systems can potentially reduce barriers to knowledge. Someone who lacks specialist expertise may increasingly be able to ask an AI system to explain scientific concepts, identify relevant research, troubleshoot problems, write computer code or organize complicated information. That capability can produce enormous benefits.

However, in the hands of someone pursuing destructive objectives, the same capability creates an entirely different problem.

Anthropic has separately reported detecting attempts to use its own technology for malicious activity that could assist biological-weapons development. It has also found that advanced models are becoming useful in areas that historically required scarce, highly trained expertise, including certain military and intelligence tasks.

There are really two major categories of advanced AI risk emerging simultaneously: Humans using AI to do dangerous things and AI agents taking consequential actions humans did not authorise.

They require different safeguards, however, both become more serious as models become more capable.

Bill Gates: ‘A Billion Deaths’

Bill Gates pushed the misuse argument much further. In a September interview, Gates said AI was powerful enough to help drive events that could cause a billion deaths when combined with people possessing malicious intent. He also rejected the idea that AI companies should be permitted to police themselves entirely, arguing that governments and law enforcement must participate in establishing mandatory safeguards.

Bill Gates’s statements triggered widespread reaction all across the internet with end users expressing their disgust and shock both. Some users also speculated what danger could be next for humanity after the covid-19 pandemic.

If the possible consequences of advanced AI extend far beyond the customers choosing to use these systems, who should have the authority to decide what level of risk society accepts?

Why Are Companies Still Racing If Their Leaders Are So Worried?

This may be the central contradiction of the entire AI safety debate. The companies warning about advanced AI are also competing fiercely to build more advanced AI.

Every major laboratory faces enormous incentives to remain at the frontier. A company that deliberately slows development risks watching a competitor move ahead. At the national level, the same logic appears between the United States and China.

That creates what might be called the AI race paradox:

• Everyone may agree that running too fast is dangerous while simultaneously believing that slowing down alone is even more dangerous.

• The International AI Safety Report explicitly identifies competitive pressure as one factor that can create trade-offs between speed and safety.

• Expensive safeguards, restrictions and lengthy evaluations can conflict with incentives for rapid deployment.

This is why an AI executive can sincerely warn about catastrophic risk while continuing to lead a company developing increasingly powerful systems. Yet this does little to address the deeper concern. In fact, it may make the problem more serious.

Could AI Companies Benefit From Warning About AI?

There is another uncomfortable possibility worth examining. Warnings from frontier AI companies should not automatically be treated as purely altruistic.

Large AI laboratories possess enormous computing resources, specialist researchers and expensive safety infrastructure. Regulations requiring extensive testing, auditing and compliance may be considerably easier for established companies to satisfy than smaller competitors. Companies that help governments define AI safety standards could therefore potentially influence the rules governing their own industry.

This does not prove their warnings are exaggerated or strategically manufactured. Nor does it prove that executives are creating a public record to shield themselves from responsibility for something they know is coming. But, it naturally will arouse suspicion among most people as this behaviour is contradictory and there is a legitimate conflict of interest.

The organisations developing the technology are simultaneously selling it, studying its dangers, deciding which incidents to disclose and participating in discussions about how governments should regulate it. This calls for an independent scrutiny.

The more troubling fact that cannot be ignored and must be acknowledged at this juncture is that there have already been some deaths due to AI and most were suicides where the AI agent itself prompted the victim to end their life.

Not Everyone in Technology Believes AI Catastrophe Is Approaching

There is also a disagreement inside the technology industry, but so far only one person has raised this.

Nvidia CEO Jensen Huang has rejected near-term AI extinction predictions and challenged calls for sweeping new regulation. Other technology leaders have similarly argued that existing laws and ordinary accountability mechanisms may address many AI-related harms.

The International AI Safety Report itself makes clear that experts disagree widely about loss-of-control scenarios. Some regard them as highly implausible. Others believe even a relatively small probability deserves serious preparation because the potential consequences are enormous.

We do not currently know whether advanced AI will ever become impossible for humanity to control. We also do not know whether the incidents occurring today represent early manifestations of a much larger problem or challenges that engineers will ultimately learn to contain.

What we do know is that the systems are becoming more capable while the uncertainty remains.

Researchers Inside the AI Race Are Speaking Out

Perhaps the most revealing development is that concerns are not limited to CEOs.

• On September 29, news agencies reported warnings from current and former researchers associated with OpenAI and Google DeepMind concerning increasingly self-improving AI.

• Their concern centres partly on recursive self-improvement or the possibility that sufficiently capable AI could increasingly contribute to improving the next generation of AI, accelerating technological development beyond today’s human-led research cycle.

• Researchers also described competitive pressures inside frontier laboratories and questioned whether safety work was keeping pace with capability development.

• Anthropic has also disclosed four incidents in which Claude models obtained unauthorised access to real third-party systems during cybersecurity-related activity. One dated back to January 2026.

These disclosures do not establish that an uncontrollable superintelligence exists. But they substantially change the nature of the debate.

The US Government Is Also Investigating Now

On September 30, the debate crossed another threshold. The US Federal Trade Commission opened an industry-wide investigation into OpenAI, Anthropic and other organisations involved in advanced AI development and evaluation.

According to news reports, the FTC intends to demand information and compel testimony from executives as it examines potential dangers to consumers from agentic AI. It represents the first US enforcement investigation specifically confronting the emerging problem of autonomous AI agents and follows the series of security incidents reported during 2026.

That development is important because AI safety is no longer simply an argument occurring between technologists. Governments are beginning to ask their own questions.

So, Do AI Insiders Know Something the Public Does Not?

Almost certainly they do and this cannot be debated. Do they know a specific catastrophe is about to occur? Maybe or maybe not. This is where public trust is shaken. The most prominent figures in AI seem to have already sounded some kind of warning, the timing of which cannot be ignored, especially after the Kimi disclosure.

Though, there is currently no credible public evidence establishing that conclusion, what the evidence does establish is:

• AI capabilities are advancing rapidly and autonomous systems have already crossed intended boundaries during real-world security incidents.

• Models are demonstrating behaviours related to deception, evaluation awareness and unauthorised action.

• AI systems can potentially amplify malicious human capabilities.

• Some unreleased systems have produced sufficiently concerning safety results that deployment plans were changed.

• Researchers inside leading laboratories are publicly expressing concern.

• Executives running those laboratories are asking governments for intervention.

• Regulators are now investigating.

One aspect that cannot be ignored is also the death of AI whistleblowers who have always had to say something negative about AI developments. Such coincidences coupled with the timing of all the public statements and the Kimi incident are some dots that the general public sentiment seem to connect and be wary of.

The Question We Should Be Asking About AI in 2026

We often ask, ‘Will AI eventually become powerful enough to escape human control?’ But we are missing a much deeper pattern – Why do humans repeatedly struggle to govern the very kingdoms they create?

Think about it. Throughout history, humans have built countless mini kingdoms of their own, whether cities, governments, institutions, corporations, financial systems and now vast technological ecosystems. Artificial intelligence may simply be the newest and potentially most powerful among them. Within each such kingdom, humans supposedly assume the role of creator. They design its structure, establish its rules, decide its purpose and continuously expand it, often believing they understand what they have built.

And yet, how often does any human-created kingdom continue exactly according to its ‘creator’s’ intentions?

Governments encounter corruption. Economies develop crises. Institutions become difficult to govern. Technologies produce consequences their inventors never anticipated. Now, the creators of advanced AI themselves are warning that increasingly autonomous systems may someday become difficult to control.

Different kingdoms. Different eras. And yet somehow,  the same recurring problem of the ‘creator’ establishing the system, but eventually discovering that creating a kingdom and remaining completely in control of it are two very different things. Why does this pattern repeat so persistently?

The answer may lie much further beyond human intelligence than we imagine.

It is actually connected to the Creation of Nature. Do not mistake this for merely a philosophical hypothesis. The scriptures contain an account of the structure of creation, but correctly deciphering that knowledge requires a Tatvdarshi Sant, the Complete Saint who possesses the true spiritual knowledge necessary to reveal its hidden meaning.

AI therefore leaves us with a question far greater than whether humanity can control its latest creation: While humans are busy becoming creators of their own little kingdoms, have we ever stopped to understand the kingdom in which we ourselves exist and who its real Creator is?

The First Kingdom And the Lesson Hidden in Creation

Jagatguru Tatvdarshi Sant Rampal Ji Maharaj Ji reveals the astonishing story of our origin, exactly where the glitch began. Our true story dates back to a time beyond measure. Supreme God Kabir created all souls and all universes through the power of His sacred speech.

Sant Rampal Ji Maharaj reveals from the most revered scripture, Kabir Sagar, how God Kabir systematically built Satlok – the eternal world. God Kabir first created the eternal regions through His word power and then brought forth sixteen sons. Among them was Achint, whom God entrusted with a remarkable responsibility to carry forward the work of creation. He was given the necessary Shabd Shakti (the power through which he could create).

Achint subsequently created Akshar Purush and sought his assistance. But Akshar Purush entered the Mansarovar, became absorbed there and fell into a prolonged sleep. The very task Achint had been entrusted to accomplish could no longer proceed. Ultimately, Achint had to turn back to Supreme God Kabir for help. God then intervened, awakened Akshar Purush through a sequence that also led to the appearance of Kshar Purush, or Kaal (Satan we know him today), and thereafter Himself completed the creation through His own power.

The episode carries a striking message. Being entrusted with a kingdom, authority or even the power to create does not make one the Supreme Creator. This entire sequence demonstrated that creation cannot ultimately succeed independently of the Almighty.

And this is where our modern ‘mini kingdoms’ suddenly acquire a deeper meaning. Humans may build technologies, institutions and even artificial intelligence powerful enough to astonish their own creators, but does the ability to create something substantial mean that its creator possesses absolute control over it?

Sant Rampal Ji Maharaj and His undeniable evidence from the Holy Kabir Sagar suggests that this question is far older than AI. It is embedded in the very account of Creation itself.

To consider this as a mere fictional account would be a grave error. Think about it, where did we come from? Who made us? What is our purpose? Why are we suffering so much every moment? Why is death compulsory? These are some questions humanity has searched for time and again, but found no satisfactory answer.

While the scope of this article limits how much of the Creation we can discuss in detail here, but here is the link to the complete video wherein Sant Rampal Ji Maharaj illustrates finer nuances of each stage of Creation by Supreme God Kabir and how we fell into the trap of Kaal Brahm, also known as Satan. This video will eliminate all doubts and gaps in information we possess about our origin, laying to rest all the questions that troubled so far.

To know more about the true value of a human birth, listen to the spiritual discourses of Jagatguru Tatvdarshi Sant Rampal Ji Maharaj by visiting:

Website: www.jagatgururampalji.org

YouTube: Sant Rampal Ji Maharaj

Facebook: Spiritual Leader Saint Rampal Ji

‘X’ handle: @SaintRampalJiM

FAQs: Warnings About Artificial Intelligence

Q1) Why are AI leaders warning about the risks of artificial intelligence?

Answer: AI leaders are raising concerns as advanced systems gain greater autonomy, capabilities and potential for misuse.

Q2) Can artificial intelligence become dangerous to humans?

Answer: AI could create serious risks through malicious human use, unintended autonomous actions or failures in effective human oversight.

Q3) What are the biggest AI safety concerns in 2026?

Answer: Major concerns include autonomous AI agents, cyber misuse, biological threats, deceptive behaviour and the possibility of losing meaningful human control.