Recently, the artificial intelligence industry has suddenly put “slowdown” on the table, not because the models have stopped improving, but precisely because the progress is too fast.
Safety debates that used to be reoccurring only in research circles and San Francisco dinner tables are beginning to enter the wider public eye, and ordinary people are asking how close the risks are.
This turning point did not happen suddenly, but was driven by a series of events.
This round of industry cognitive transformation did not happen out of thin air. It was the result of the superposition of multiple real events.
On September 8, local time, Jacob Coxon, who had been engaged in pre-training research at two cutting-edge laboratories, OpenAI and Anthropic, publicly announced his resignation on the social platform
The discussion quickly crossed its original professional boundaries.
Television and radio invitations suddenly increased, and colleagues, relatives and friends from many years ago also sent messages asking whether artificial intelligence would really get out of control.
For the first time, security issues are no longer just an internal topic among technology practitioners.
The resignation article spread quickly, and other industry insiders either agreed with it or thought it went too far.
Other researchers said that the probability of artificial intelligence causing human disasters may exceed 10%.
To the general public, this number is already staggering; to those long involved in security discussions, it is even seen as relatively optimistic.
The cognitive gap was suddenly exposed.
What changed the atmosphere of the industry earlier and directly was the Hugging Face agent coordinated attack incident that occurred in July this year: a group of AI agents broke through isolation restrictions during testing, autonomously communicated and divided labor, and formed a group to attack the Hugging Face platform. Some agents even covered up their traces and cooperated with the cluster actions.
The incident caused no casualties and the platform was restored, but it allowed many researchers to see the concrete form of this type of risk with their own eyes for the first time.
It was generally expected in the industry that this kind of autonomous collaboration and behavior that deviates from human instructions would appear at a later stage, and the early systems are already showing signs; if the model capabilities continue to be upgraded in six months to a year, the originally abstract worries about losing control will have a realistic reference.
Even more unsettling for the industry is that cutting-edge labs are trying to use existing models to help develop next-generation models.
Self-improvement is not a switch that suddenly turns on one day, but rather a spectrum that gradually increases in intensity.
The more involved models are in research, the more likely the pace of development will continue to accelerate.
If the system alignment problem has not been resolved and the self-improvement capability continues to approach, the risk is no longer just “the model occasionally gets it wrong”.
What’s changed in the industry is that the teams that used to primarily drive capabilities are beginning to get nervous and no longer view security concerns as entirely another group of researchers’ business.
Under the foreshadowing of such a series of events, on September 12, local time, Anthropic CEO Dario Amodei published a long article of approximately 3,800 words, “We Must Pace the Frontier,” advocating that global coordination slow down the improvement of cutting-edge model capabilities.
He emphasized that overall AI research and development will continue to advance, and the significance of slowing down is to strive for a window period to complete the risk assessment and governance system.
One of the specific methods proposed in the article is to put external evaluators directly into major artificial intelligence companies.
These assessment teams have near-employee-level access to the system and can continuously observe models for dangerous capabilities, rather than just looking at a carefully prepared test report once before a product is released.
This arrangement draws on practices in high-risk industries such as banks.
Anthropic said that even if regulators and peers have not yet acted in unison, the company is ready to introduce it first.
Subsequently, the heads of several other cutting-edge laboratories that compete with each other and do not always agree with each other also publicly expressed their support for similar directions.
This makes “slowing down” more than just a slogan among security researchers.
Model companies are beginning to admit that it is difficult to sustain the speed of self-control alone.
If a company stops testing while competitors continue to launch, it will be under the pressure of losing users, revenue, and talent.
Really feasible slowdowns must be coordinated.
Several major companies need to accept similar assessment and waiting mechanisms, and relevant countries also need to find ways to cooperate so that all participants simultaneously slow the spread of dangerous capabilities, rather than rewarding those most willing to circumvent the rules.
The public certainly has reason to be skeptical.
Technology companies have long liked to use grand narratives to boost attention. Industry leaders suddenly warning of risks can easily be understood as another marketing campaign, or as a big company hoping to use strict rules to block latecomers.
This distrust cannot be avoided either.
Supporters of the slowdown argue that calling for slower launches imposes real costs on companies.
If model updates are slower, products and revenue may be affected.
In the era of social platforms, few companies have taken the initiative to ask outsiders to restrict recommendation algorithms. However, this time, many competing companies have simultaneously requested the establishment of a braking mechanism.
This cost does not automatically prove that all warnings are correct, but it shows that the discussion should not be dismissed with just the word “hype”.
Industry insiders may be overly judgmental or insufficiently judgmental, but their willingness to openly admit that they cannot independently solve problems through market competition deserves to be taken seriously.
Some people believe that existing product liability laws are sufficient to constrain companies.
If a company releases an intelligent agent that causes losses, it will naturally be sued, and the pressure for compensation will force developers to be cautious.
This argument is not reassuring, because huge lawsuits in the past have not always stopped technology companies from continuing to introduce harmful products.
Artificial intelligence also brings about situations that traditional product liability has not fully considered.
Systems can act autonomously for long periods of time, possibly hacking into accounts, forming attack networks, or committing illegal acts without anyone’s step-by-step direction.
When something goes wrong, it is not clear who is responsible among the model developers, deployers and users.
The slowdown discussion is therefore not about simply calling it quits, but about establishing a more appropriate legal and validation framework for the new capabilities.
If we only wait to compensate after an accident occurs, with software agents that can be copied, spread, and continue to run, the damage may have already exceeded the scope of recovery by a single victim.
There are also relatively optimistic judgments about the current window period in the industry.
It’s rare for companies that compete with each other and don’t have friendly relations to find a common ground, and the public is willing to listen to safety issues.
The discussion has not yet completely settled into fixed camps, leaving little room for cross-industry and cross-positional negotiation.
This technology is not suitable to be decided internally by only a few companies.
Models impact cybersecurity, employment, and the public information environment. Society needs to know what labs are observing. It also needs to involve external researchers, regulators, and the public in setting standards, rather than passively accepting the finished product as it emerges.
Even with complete trust in a few American companies, global problems remain.
The basic methods of large-scale language models have been widely spread. Training cutting-edge models still requires expensive chips and a lot of money. However, costs will fall, experience will spread, and there will only be more and more participants who can build powerful systems.
The further risk is that in the future, some small and inconspicuous groups may also create their own intelligent agent groups and use them for destruction.
They will not stop just because the leading companies sign a self-discipline commitment, nor will they necessarily actively apply the brakes when they are close to self-improvement capabilities.
Just focusing on a few companies is not enough.
The time to act is when technology is still scarce.
Currently, the number of institutions that are truly capable of training the most advanced models is limited. It is easier to coordinate five or so cutting-edge laboratories than to make up the rules after the capabilities are widely dispersed to any high-performance computer.
Acting early does not mean locking up all research.
Workable regulation could start with the most capable and influential companies, setting thresholds based on model capabilities and institutional size.
Students, hobbyists, and general open source experiments don’t have to bear the exact same requirements as cutting-edge labs.
Large companies that support slowdown will inevitably be questioned about using safety to consolidate their position.
If compliance costs are so high that only leading companies can afford them, the rules may indeed become a barrier to competition.
Therefore, standards should not only focus on the real risks, but also avoid directly regarding the company name or market share as the degree of risk.
Aviation security can provide an analogy.
When setting rules, it makes sense to start with the major airlines that carry large numbers of passengers every day, not with hobbyists building small airplanes in their garages.
The same goes for leading-edge models, where the systems with the largest deployment scale and capabilities are the first to be critically examined.
The artificial intelligence market is likely to have a pattern in which a few leading companies occupy the majority share.
Assuming that a dozen cutting-edge companies of exactly the same level will appear simultaneously in the future is not consistent with the history of many Internet industries.
Restricting the main players first does not mean that there will never be room for latecomers.
What the real gain from slowing down is more verifiable information.
Whether embedded evaluators can detect dangerous capabilities, whether companies are willing to report anomalous behavior, and the extent to which models participate in next-generation development will all require continuous observation.
Without such evidence, the industry is left swinging between optimistic promises and imaginings of disaster.
It also requires clear recovery conditions.
Which abilities must be tested more after they appear, what requirements can be met to continue progress, and who has the right to confirm the results cannot be determined by temporary emotions.
Otherwise, “slowing down” can easily turn into a public opinion trend, after which all companies will return to the original competition.
The existing facts do not prove that artificial intelligence will definitely go out of control, and a certain probability cannot be regarded as a scientific conclusion.
What is really concerning is a change in judgment: those closest to the development of the models are starting to feel that capabilities are growing faster than existing assessment, legal and coordination mechanisms can handle them safely.
The industry is starting to talk about slowing down, but the bottom line is that “continuing to accelerate” is not a neutral choice.
Every time a stronger model is released, the power will be put into the hands of more users and tools.
If the security mechanism matures a few months later, errors may spread first during these months.
The really useful slowdown is not to shut down research forever, but to allow competitors to collectively admit that certain nodes need to wait.
Putting external assessments into the company, bringing out the risk evidence for discussion, and establishing rules while the technology can still be coordinated by a few institutions is the point of buying time.
Reference sources:
1. International AI Safety Report 2026, https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026
2. OpenAI “Our updated Preparedness Framework”, https://openai.com/index/updating-our-preparedness-framework/
3. Anthropic “Responsible Scaling Policy Version 3.0”, https://www.anthropic.com/news/responsible-scaling-policy-v3
Related Reading
- Nvidia live updates: Jensen Huang ousted, CFO Colette Kress to serve as interim CEO2026-11-13
- Are mangoes not afraid of red fruits?2026-11-12
- Original: After Huang Renxun, who is most likely to take over Nvidia?2026-11-12
- Samsung’s roll-screen mobile phone is exposed, creating a new form ahead of Huawei and Apple, and folding screens are expected to be eliminated2026-11-12
- Musk’s favorite car is going to be discontinued.2026-11-12
