AI Infrastructure & Hardware Economics

Why a brand-new GPU rack can be worth almost nothing within two years

The most expensive assets in modern AI infrastructure depreciate at a rate that would make a car dealer nervous. There is no liquid second-hand market for used GPU clusters, and the machines that dominate training today will be structurally obsolete before their loan terms are paid off.

• Flagship AI accelerators such as NVIDIA's H100 retail for roughly $25,000 to $40,000 each, and a single training rack can run into the tens of millions of dollars.
• In a healthy secondary market, a 2- to 3-year-old accelerator trades at about 50% to 70% of new price — but that rule barely applies to purpose-built cluster hardware.
• Resale value now drives every major AI-infrastructure deal, yet there is no standard way to price a used rack as a unit.

The problem is not that the chips stop working. A two-year-old H100 is still a powerful GPU. The problem is fit: each generation is engineered for a specific memory bandwidth, interconnect fabric (NVLink), and software stack, all optimized together for one class of workloads. When the next generation ships, the whole rack becomes harder to re-purpose rather than simply slower.

Cloud providers and startups buy racks on credit or long-term leases precisely because the upfront cost is prohibitive. But without a deep resale market, exiting that debt means selling to a narrow set of buyers — other cloud providers, crypto miners repurposed for inference, or secondary brokers — who all know the hardware's clock is ticking. That thin buyer base is why published residual-value studies repeatedly find the secondary market far less predictable than the headline percentage suggests.

There is also a structural asymmetry between training and inference. A cluster bought to train foundation models can struggle to earn back its cost once its owner pivots to lighter inference workloads. Idle GPUs in large facilities can bleed tens of thousands of dollars a month in pure fixed cost, even before electricity and staff are factored in.

The lesson for anyone financing AI infrastructure is counterintuitive: the hardware is less like a fixed asset and more like a consumable. Treat a GPU cluster as something you will have to replace on a multi-year cadence, not as a long-lived capital investment. Depreciation, not performance, is the real constraint on the economics of frontier AI.

Knowledge takeaway: flagship AI accelerators cost tens of thousands of dollars each with racks reaching tens of millions; published residual studies peg 2-3 year-old GPUs at roughly 50-70% of new price, but thin resale markets make real values far less predictable; idle enterprise GPU racks can lose tens of thousands of dollars per month in fixed costs.