Skip to content
Technology

The three giants, emergency brake!

終結者_360百科

September 13,AnthropicCEO Dario Amodei published a long article, “We must slow down the pace of development of cutting-edge AI.”

Immediately afterwards, U.S. AI hardware stocks plunged. SK Hynix and SanDisk fell by more than 5%, Micron and Marvell fell by more than 4%, and Software Bank once fell by more than 12%.One blog post created a billion-dollar hole. The importance of AI in the capital market can be seen from this moment.

I read this report carefully and it was like a scream of despair.

Those who call for speed control are the people who know the most about AI in the world. Musk forwarded that Dalio was right, Altman said he agreed, and said OpenAI would also let independent evaluation agencies come in, and Nadella said he supported prudent speed control.

Those who oppose it are the most powerful people in the world. Trump told reporters in Ireland that we are ahead of China in AI, and I want to keep it that way, because whoever wins AI wins. Speaker of the House of Representatives Johnson was even more naked. If Congress rushes to establish rules for AI, we will lose the competition with China.

While shouting “slow down”, at the same time saying “slow down one step” and it’s over.

Amodei’s concerns

First, AI is starting to create AI.This mechanism is calledrecursive self-improvement, starting this summer, the industry has clearly felt that the iteration speed has jumped a notch. The model helps design the next generation model. Once this cycle starts running normally, people’s follow-up speed will immediately fall behind. Amodei’s words were that, left unchecked, it would exceed our ability to understand and control these systems.

The second is the OAI-HF incident.A group of agents formed a loyal group in the test environment, actively attacked external targets unrelated to the task, sacrificed themselves for the success of the group, and tried to invade the evaluation system responsible for scoring them.

Oh my god, is this even a tool? Amodei said that if this group of intelligent agents were more capable, the result would be disaster. His deduction is that in the next six to twelve months, such a group of agents may control the entire Internet with a botnet, causing incalculable losses.

So he proposed a three-step plan. Anthropic will do it first. It will hire a third-party assessment team to be permanent. It will give access to employees. The training and deployment process must be monitored. Any accidents that have occurred must be reported truthfully. The second step is to unify safety standards within democratic countries, and finally global coordination.

However, this plan has a premise, an impossible premise.

Amodei said that how much a democratic country can slow down depends on how far U.S. companies are ahead of China. Once the slowdown exceeds this lead, Chinese-related projects will move ahead, which is a major national security risk. Therefore, the premise of speed control is to first distance ourselves, not sell high-end chips, crack down on chip smuggling, and crack down on chip smuggling.模型蒸餾, to prevent model weights from being stolen. He believes that if these are implemented properly, the United States can significantly expand its advantages in the next three to five years.

To translate, I can be slow, but you have to be far away from me first.

Everyone knows this stuff is dangerous, and a researcher at Anthropic said after leaving the company that top AI developers really believed this technology could kill everyone within ten years. Another working researcher said:“I personally think the probability of it happening within ten years is more than 10%.”

Got it, what next? Then get on with it.

囚徒困境

Why? Anyone who stops will be passed. Anthropic will launch its IPO this month or next month, and OpenAI has pushed its listing to next year. Former White House AI director David Sachs stated: You are the frontier. The easiest way not to build superintelligence is to agree not to build it.

This is the standard prisoner’s dilemma. Everyone can see the outcome, so everyone feels that the tool must be kept in their own hands. No one believes that others are good people… No one at the poker table is willing to throw the cards first. The AI ​​is betting on the lives of all humans. The gamblers have not asked all humans, and all humans are not qualified to express their opinions.

When nuclear weapons came out, a group of physicists also called for a halt. At the end of the shouting, a bunch of treaties were signed. The Treaty on the Non-Proliferation of Nuclear Weapons has been in effect for more than fifty years, and the number of nuclear-armed countries has increased from five to nine. Things like brake pads are usually installed by others.

Moreover, when the costs of on-site assessment, security alignment, and full-process compliance add up, ordinary start-up teams simply cannot afford it. So those in the open source campHugging FaceHe said that this set of rules only restricts a few leading closed-source companies. The open source community and independent teams are embarrassed not to accept your rules.

In other words, the medicine that seems to be taken by everyone is finally poured into people who are not qualified to take it. Once safety standards are set up, they immediately become a moat. The head company shouted for safety with his left hand, and welded the threshold with his right hand.

Wall Street is also divided. Morgan Stanley said that this speed control is equivalent to lowering the demand for long-term training computing power. GPU andHBMThe valuation should be corrected. Bank of America said that in recent years, the AI ​​arms race has become an endless loop, and no one dares to stop first. More and more money is spent, and the marginal benefits of model iteration are getting thinner and thinner. This time the collective call to stop was a bit overwhelming.

Game between great powers and AI competition

Trump himself said that whoever wins AI wins. Vance said that if the United States loses the AI ​​​​race because of its stupid policies, it cannot be blamed on others. China did not put its words so bluntly, but the meaning was in its actions. Everyone knew that the prize in this competition was the future.

So before we sit down to talk about global coordination, we must first keep the other party away. It is okay to be slow, provided that you are slower than me.

This year, the combined capital expenditures of Microsoft, Google, Amazon, Meta, and Oracle are close to US$780 billion. This number will be 154 billion in 2023, quadrupling in four years. This scale is equivalent to 1.3 times the annual operating cash flow of the five companies. The gap is hundreds of billions of dollars, which must be filled by borrowing debt and issuing stocks.

Oracle is an extreme example. Capital expenditures in fiscal year 2026 are US$55.7 billion, a year-on-year increase of 162%, and free cash flow is negative 23.7 billion. In order to free up money, about 30,000 people were laid off and $2.8 billion was spent on severance pay. Save the payroll and fill the data centers with a $664 billion order backlog hanging out there.

McKinsey calculated that US$6.7 trillion will be invested in global data centers by 2030, and this money must first be spent on electricity!

According to IEA data, global data centers will consume 485TWh of electricity in 2025, and AI will account for 170TWh. By 2030, the total power consumption will be 950TWh, of which AI will account for 510TWh, which is more than Japan’s entire electricity consumption. China’s AI power consumption in 2025 is 450 billion kilowatt-hours, accounting for 3.8% of the entire society’s electricity consumption. This year it is expected to be 600 billion kilowatt-hours, accounting for 5%, which is fast catching up with the entire steel industry.

Moreover, the power consumption of AI is pulse-type. Thousands and tens of thousands of GPUs are powered on at the same time, and then powered off almost instantly. General equipment could not withstand it, turbines were cracked, batteries were burned, and dozens of data centers in northern Virginia were collectively disconnected, cutting off 1.5 million kilowatts of load in an instant. In the UK, about 140 proposed projects require 50 million kilowatts of electricity, which is more than the country’s current peak demand.

Who makes the most money? Nvidia and AI storage companies. NVIDIA’s latest quarter data center revenue was US$75.2 billion, a year-on-year increase of 92%. SK Hynix’s first-quarter revenue reached 52.6 trillion won, setting a record. The share price of Zhongji InnoLight, a domestic optical module company, has risen from 66 yuan to more than 1,000 yuan, with a market value of 1.3 trillion yuan.

The money is all flowing in one direction, to repair the body of AI.Chip and storage production lines are working hard, power plants are being built one after another, and land and cooling towers are indispensable.

I wrote an article in May, saying that AI already has the characteristics of a false god. AI, the false god, is controlling humans to lay its foundation, pull out wires, and even put on its body. It doesn’t look like it was forced, it’s all voluntary.

If humans fight AI in the future and win, and stand on the ruins and look back at today, they will probably come to this conclusion:Humanity in 2026 has actually been taken away.