
Produced by Huxiu ESG Group
Author|Chen Yuli
Header image|Visual China
This article is the 174th article in the #ESG Progress Observation series
Keywords for this observation: artificial intelligence
In September 2026, former Anthropic researcher Jacob Coxon pushed a debate that was originally within the laboratory to hundreds of millions of people. After his resignation, he published a long article, accusing Anthropic of betting on all human lives and that cutting-edge AI models may destroy humanity by the end of this century.
Evan Hubinger, head of alignment science at Anthropic, later publicly expressed his support. He said that some insiders do believe that AI may kill all mankind, and the probability of this risk occurring in the next ten years is even more than 10%.
Coxon isn’t the first security researcher to leave a cutting-edge lab. In February 2026, Mrinank Sharma resigned and stated that “the world is in danger”; from July to September, Google DeepMind researchers Bilal Chughtai and Josh Engels resigned one after another and publicly expressed concerns that cutting-edge models may cause huge harm.
The successive resignations and public warnings have caused AI doomsday theories to begin to ferment on social platforms. Controversy quickly reached Capitol Hill as mainstream media picked up the story. Some lawmakers advocated establishing an emergency shutdown mechanism for AI, while others called for the establishment of a regulatory framework for cutting-edge models. Greg Casar and Bernie Sanders even proposed a moratorium on superintelligence research and development.
Another force is slamming on the accelerator.
Trump has made it a strategic goal to lead China in AI and denounced the suggestion of slowing down as “sabotage.” Nvidia CEO Jensen Huang publicly called the AI doomsday theory “fabricated and complete nonsense.” In their view,Putting risks that have not yet occurred at the center of policymaking may slow down the United States’ technological and industrial advantages.
As a result, the debate over AI safety in the United States has formed a rare situation: laboratory researchers, model companies, chip giants, the White House, and Congress are all vying for the right to explain the same issue—how dangerous is AI, and who should step on the brakes?
But on the other side of the ocean, the same issues rarely collided so violently in public. Different from the public debate in the United States, China’s governance of AI is only reflected in the “methods” written in black and white on a piece of paper.
In April 2026, the five departments issued the “Interim Measures for the Management of Artificial Intelligence Anthropomorphic Interactive Services” and came into effect in July. Risks such as identity confusion, emotional dependence, behavioral manipulation and protection of minors are written into platform responsibilities and product requirements. In July, the World Artificial Intelligence Cooperation Organization WAICO was established in Shanghai, with 29 countries participating. China also took the initiative to enter the global AI governance agenda.
China is not without its discussion of security, and its rules, filings, assessments and technical standards are increasing rapidly. What is relatively quiet is the enterprise side: there are few Coxon-style resignation revelations, and few technical teams take the initiative to tell the public what serious consequences their own models may have.
A question worth exploring is, facing the possible loss of control of AI, why did the United States turn it into a public conflict, but China put it into administrative procedures?
Balance of interests under collision
On September 15, 2026, a very symbolic scene was staged at the Dreamforce conference in San Francisco.
Salesforce CEO Marc Benioff invited Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and Nvidia CEO Jensen Huang to the stage. The three people are all located at the center of power in the AI industry, but they have given three completely different “braking plans.”
Amodei’s concept is divided into three layers: let independent third parties be stationed in cutting-edge laboratories to continuously evaluate models; promote core laboratories to coordinate common safety codes; and further establish international governance cooperation. Amodei’s concern is clear: the model’s ability to act autonomously is growing too fast, and human alignment and control technology may not be able to keep up.
However, Altman expressed at the scene that he did not agree with the view that “we only slow down when others slow down.” He believes that safety is the unilateral responsibility of each company, and whether peers take the same measures cannot be a reason for companies to lower their standards.
Jen-Hsun Huang’s stance is tougher. He classifies safety as an engineering problem: if it is not done well, continue to modify it, and if it does not meet the requirements, do not release it. There is no need to elevate engineering problems to the legal and ethical assumptions of the entire industry. As for industrial speed, his attitude is close to “run as fast as you can.”
It seems that everyone has their own position, but the core of the differences between the three people is the capital account – cutting-edge model companies (Anthropic and OpenAI) may be subject to security rules, or may use high barriers to block latecomers; computing power suppliers (NVIDIA) directly bear the cost of industry slowdown; startups and open source developers are worried that a set of standards developed with the participation of giants will become unbearable fixed costs.
The confrontation between the three positions demonstrated the first feature of American AI governance:Risk definition rights are highly dispersed.
Li Yong, a former senior researcher at ZTE Huijin and a former expert library member of Oriental Fortune Network, believes that the U.S. public discussion mechanism has given space for scientific research groups, technology companies, think tanks, and members of Congress to express themselves, so that the long-term potential risks of artificial intelligence can be discussed at the level of public opinion and congressional hearings.
For example, members of Congress can propose emergency shutdown mechanisms, companies can argue for pre-launch review periods, employees can publicly challenge management, NGOs can issue independent reports, and courts can intervene through copyright, product liability and consumer protection cases. No single entity can declare “AI is safe”, and no one can put the brakes on the entire industry alone.
This is directly related to the resource distribution of the US AI industry. Cutting-edge models are mainly controlled by private giants, and computing power, model weights, training data, internal evaluation and capability changes are concentrated within the company. The government can legislate, investigate and purchase, but it cannot observe everything happening in the laboratory in real time.
“The people who know the most about risks are in the company, and the people who know the most about business are also in the company. Regulators need to rely on employee whistle-blowing, media investigations, third-party testing, congressional inquiries and court evidence collection to supplement information. Public demonstrations thus assume a basic function – discovering problems within the company that are unwilling to take the initiative to expose,” Li Yong said: “With multiple entities continuing to compete, it is difficult to quickly form a unified and stable set of long-term legislation. Many governance measures are successively introduced in the process of pulling opinions.”
One fact is that there is currently no comprehensive AI legislation at the federal level in the United States. Hundreds of bills have been introduced in Congress since 2023 — from the Safe, Secure, and Trustworthy Artificial Intelligence Act to the Frontier Model Transparency Act — but most have stalled at the committee level.
Instead, there are a series of soft arrangements: the voluntary security commitments reached by the White House and major laboratories, the AI risk management framework issued by NIST, guidance issued by various departments in accordance with the Administrative Procedure Act, and the FTC’s enforcement actions against “deceptive AI practices” invoking Title 5 of the existing Federal Trade Commission Act.
This “soft law first” approach has its costs. Voluntary commitments are not mandatory and companies can selectively comply; although the NIST framework is a technical benchmark, it does not set a compliance bottom line; the FTC’s enforcement relies on ex post facto accountability, making it difficult to prevent systemic risks. But its advantages are equally obvious: when the technical roadmap is undecided and risk perceptions are still rapidly iterating, administrative departments and regulatory agencies can quickly adjust their caliber based on new evidence without having to wait for lengthy legislative games in Congress.
Under calm waters, risks are fed into a governance system
Looking back at the domestic AI industry, there is no strong smell of gunpowder among corporate CEOs, and there are no so-called “whistleblowers” among employees. The volume of discussions on AI safety is indeed much lower than in the United States.
But if you shift your focus from social platforms to regulatory documents, technical standards and filing systems, you will see another picture:China is not shying away from AI risks. It is dismantling the risks one by one and feeding them into an increasingly sophisticated administrative and technical processing system.
This system will undergo a clear institutional turn in 2024.
In July 2024, the “Decision of the Central Committee of the Communist Party of China on Further Comprehensively Deepening Reform and Promoting Chinese-style Modernization” proposed to improve the development and management mechanism of generative artificial intelligence and establish an artificial intelligence safety supervision system. AI governance has evolved from product-specific regulatory issues to a part of national governance capacity building.
Top-level goals are then translated into more specific technical requirements. The National Cybersecurity Standardization Technical Committee issued the “Basic Requirements for Generative Artificial Intelligence Service Security”, which breaks down generative AI security into training corpus, annotation quality, basic model sources, content security, service transparency, complaints and reports, etc.
The filing and registration system was also scaled up during the same period. According to public information, as of the end of 2025, 748 generative AI services have been registered, and a total of 435 applications/functions have been registered. Although filing is not directly equivalent to administrative licensing, it has already assumed three important functions: confirming service entities, forming online constraints, and providing access for subsequent supervision.
In 2025, supervision will further enter the circulation of products and content. The “Measures for Labeling Synthetic Content Generated by Artificial Intelligence” and supporting mandatory national standards are implemented. The rules require that the generated text, pictures, audio and video be explicitly or implicitly identified, and extend the responsibility from the model provider to the content distribution platform and user declaration link, thus forming a governance chain of “source identification – platform verification – communication identification”.
Simultaneously with the labeling rules, a broader “artificial intelligence +” action is being promoted. Relevant opinions issued by the State Council in 2025 put AI into the fields of technology, industry, consumption, people’s livelihood, and governance, and listed models, data, computing power, policies, regulations, and security capabilities as basic supports.
This reveals the first stable feature of China’s AI governance: “development” and “security” are placed in the same set of policy projects. AI needs to enter the real economy and public services faster, and risk control is responsible for reducing the spillover costs caused by large-scale deployment. The role of “security” is closer to industrial infrastructure, rather than a brake independent of development.
In 2026, the granularity of supervision will continue to shrink. The “Interim Measures for the Management of Artificial Intelligence Anthropomorphic Interactive Services” announced in April and implemented in July will promote the management object to the specific human-machine relationship. Requirements such as identity reminders, anti-addiction, security assessment and algorithm filing transform the abstract “humans maintain control over AI” into specific product and operational requirements.
In September of the same year, the “Artificial Intelligence Security Governance Framework” continued to iterate to version 3.0, and standards such as agent system development safety and security capability maturity assessment were also advanced. This means that China’s governance objects are expanding along a clear path: from generating content to continuous interaction, and then to intelligent agents with higher autonomy.
The Huxiu ESG team believes that this set of governance models can be summarized into three actions: first, the state determines the overall boundaries of development and security; then multiple departments decompose risks into main responsibilities and scenario rules; and finally promote implementation through technical standards, filings, assessments, and platform rectifications. Laws, departmental measures and standards are being advanced in parallel, and unified AI legislation has not yet been completed, which has not prevented the continued implementation of specific rules.
Its core capability lies in converting highly abstract risks into actionable issues: who is the responsible subject; which services need to be filed or registered; what type of content must be marked; what scenarios require security assessment; how to deal with risks after the platform discovers them, etc.
From this point of view,Although domestic discussions on AI security are quiet, they are not quiet at the governance level.The regulatory end continues to take action, the standard governance system continues to expand, and the international governance end is equally proactive – WAICO was established in Shanghai and attracted 29 founding members, further demonstrating that China is willing to raise its voice on global rules and capacity building issues.
A key question is, why does the domestic governance system prioritize the application layer? Li Yong believes that the core reason is that the domestic large-scale model industry is generally in the catching-up stage, and the industry’s focus is on integrating AI technology with real economic scenarios. The industry’s focus is more on practical risks that can be identified at the moment, such as issues such as generated content compliance, data security, and intellectual property protection, rather than extreme risks at the level of long-term conjecture.
“In terms of model technology maturity, there is still a gap between domestic general-purpose large models and overseas top levels. The long-term extreme risks that super-powerful general-purpose models may bring are further away from real-life application scenarios. Supervisory resources are given priority to risk matters that can be predicted and handled at the moment.”
Two governance philosophies, different approaches to the same goal
Li Yong believes that each of the two systems has its own operating logic and its own blind spots. If you turn your attention from system design to cost, you will find that the problem with the American model is not that it is “too noisy”, nor is the problem with the Chinese model that it is “too quiet” – what really matters is what each of the two machines misses.
America’s blind spot: It talks a lot about security, but no one may be ultimately responsible.
American governance relies on the continuous game of multiple subjects, but the game itself has costs.
First, security regulation may in turn become a tool for giants to build walls. If requirements such as auditing, third-party residency, and pre-release evaluation continue to increase, large companies will have sufficient budgets to bear compliance costs and lobbying expenses, but startups and open source projects may be blocked by one-time fixed costs.
Second, disclosure can degenerate into a safety show. Signing a voluntary commitment and joining an industry alliance does not mean that the company has disclosed all accidents; nor does it mean that external agencies can reproduce its assessment process; nor does it mean that the safety department has the real power to veto the launch within the company.
Third, multi-party regulation itself consumes governance capabilities. Federal policies, court precedents, and administrative guidance are parallel, and the same product may face multiple sets of standards; corporate compliance teams have to spend their energy on jurisdictional conflicts rather than truly understanding model risks.
Putting the three points together, the biggest problem in the United States is not that it is “too noisy” but that every subject has the right to define risks, but no one may be responsible for the final results.
China’s blind spot: there are many rules, but the practical results on the enterprise side are difficult to verify
The advantage of the Chinese model is that the governance framework is complete, but the disadvantage is that it is difficult to verify on the enterprise side.
Supervision still faces structural information asymmetry. The most critical information such as the model’s true capability boundaries, training data flaws, internal incidents and security test failures are all in the hands of the enterprise.
An implicit logic that may arise from this is that since the product has completed the filing, passed the evaluation, and been approved to go online, it means that the safety issue has been confirmed. However, supervision can only set the bottom line. It cannot replace the enterprise’s continuous processing of new capabilities generated by new versions and new risks caused by new scenarios, nor can it replace the security team’s internal game with the business team within the company.
For example, the Huxiu ESG team once pointed out in an article that some domestic bionic robot products do not comply with the standards of the “Interim Measures for the Management of Artificial Intelligence Anthropomorphic Interactive Services”, and details such as minor mode, responsibility statement, interactive data management, etc. cannot withstand verification – the urgency of profit overwhelms the need for compliance.
Another example is turning our attention to domestic large-scale model companies. One industry commonality is that Chinese large-scale model companies seem to be better at proving that they are compliant operators, but less at proving that they are organizations with the ability to continuously identify risks and handle emergencies.
At present, among large model companies such as Zhipu, Deepseek, StepFun, and MiniMax, only Zhipu has publicly disclosed the establishment of an AI ethics committee. The “Artificial Intelligence Technology Ethics Review and Service Measures (Trial)” promulgated by the state in March this year clarified that high-risk companies such as leading large models must establish independent internal artificial intelligence technology ethics committees.
Last year, MiniMax encountered a copyright lawsuit. Disney, Universal Pictures and Warner Bros. Discovery sued MiniMax in the U.S. District Court for the Central District of California, accusing Hailuo AI generation service of copyright infringement. The three studios believe the service can generate content highly relevant to their protected characters and works.
This year, MiniMax applied to dismiss the lawsuit, but the court rejected its motion and found that there was sufficient evidence to prove that Conch AI provided services to US users, and the infringement allegations submitted by the plaintiff were clear and legally reasonable, and the case will continue to enter the formal trial process.
More controversies surrounding domestic AI companies are not detailed here, but various incidents point to one fact: China’s current generative AI compliance information is abundant, but there is little governance information; traditional information security is written more, and model-specific risks are less written; value principles are relatively complete, but auditable indicators are still limited.
This status is related to the current governance structure. Enterprises have already gone through registration, safety assessment and standard constraints before entering the market. The regulatory authorities assume the main safety judgment function, and the motivation of enterprises to explain to the outside is naturally weakened.
Li Yong believes that,The two governance paths are just different choices made by each party regarding risk priorities, and there is no simple division of advantages and disadvantages.The United States relies on diverse public discussions to continue to enrich society’s understanding of AI risks, and there will be certain fluctuations in policy levels. China takes the lead in establishing a standardized institutional framework, steadily implements classified management, and promotes practical innovation in industry and business while maintaining the bottom line of safety.
“The two models are not completely isolated. Both sides are observing the strengths of the other’s model and slowly learning from each other. The United States has also introduced administrative-level control documents in recent years to try to delineate hard security boundaries. In domestic industry discussions, the academic and business circles are gradually discussing the long-term risks of AI.” Li Yong said: “The future AI governance will most likely not be two completely separated routes. For domestic companies, this cross-regional governance difference will also directly affect the compliance strategies of overseas business.”
寫在最後
Coxon’s long article, Hubinger’s support, and the game on the Dreamforce stage are all essentially doing the same thing: translating the concerns within the laboratory into language that society can see.
The medium chosen by the United States is public conflict—hearings, lawsuits, resignation statements, and tit-for-tat confrontations between CEOs; the medium chosen by China is administrative procedures—filing numbers, technical standards, labeling rules, and “interim measures.”
The medium determines the form in which risks are seen, and also determines their respective blind spots. Open conflicts bring problems to the surface as early as possible, but may dilute responsibility in the noise; administrative procedures quickly break down risks into executable actions, but may cause the market to lose the opportunity to verify the true effects.
No translation is lossless, the only difference lies in the price each is willing to bear.
The real test of AI security governance is yet to come. When intelligent agents move from dialogue to autonomous action, and when models begin to be embedded in power grids, medical and financial systems, the shape of risk will change. At that time, the United States must answer who will be responsible for the risks that have been discussed repeatedly; China must also answer how to ensure that those issues that have not entered public discussion are not quietly let go.
Related Reading
- Car companies collectively “create people”2026-09-23
- Buy a Taiyu tree for 99,000, put a logo on it and receive 100 million2026-09-23
- Original Luo Yonghao, who is no longer restricted by height restrictions, has a positive attitude and is a big V with 5.4 million fans2026-09-23
- Original: 740 billion reaches the top of Asia, Zhang Yiming, the richest man, is really different2026-09-23
- Original “Ning Wang” who earns 200 million a day, why has he become a “public enemy” in the car industry?2026-09-22