AI Data Centers: Power, Cooling and Custom Silicon
The AI boom is now an energy and infrastructure story. Inside the gigawatt-scale buildout.
Key takeaways
- AI has turned data centers into an energy-infrastructure story; projects are now planned in hundreds of megawatts to gigawatts.
- AI racks draw roughly ten times the power of traditional racks, forcing a shift to direct-to-chip liquid cooling.
- Power availability and grid interconnection queues are now the gating factor for many new sites.
- Hyperscalers are signing deals for nuclear, renewables and gas generation, including restarting and extending existing reactors.
- Custom silicon and efficiency gains improve tokens per watt, but total demand keeps rising.
For decades, the data center was background infrastructure — rows of air-cooled servers drawing a few kilowatts per rack. The AI boom has transformed it into one of the most capital-intensive construction programs in the world. Today the binding constraint on AI growth is often not chips, but whether there is enough electricity, cooling and grid capacity to plug them in.
How much power are we talking about?
The International Energy Agency estimated that data centers consumed around 415 TWh of electricity in 2024 — roughly 1.5% of global electricity use — and projected that this could more than double to around 945 TWh by 2030, with AI the main driver. Growth is highly concentrated: the United States and China account for most of it, and in some regions data centers already use a large share of local power.
Individual projects have grown by orders of magnitude. A large cloud data center a decade ago might draw tens of megawatts. Today's AI campuses are announced at hundreds of megawatts to multiple gigawatts — comparable to the output of large power stations. Projects such as the Stargate initiative announced in 2025 and xAI's Colossus clusters illustrate the scale.
Inside an AI rack
A rack-scale system like NVIDIA's GB200 NVL72 draws on the order of 120 kW. Air simply can't carry that much heat away efficiently, so the industry has moved to direct-to-chip liquid cooling: cold plates sit on top of GPUs and CPUs, and coolant distribution units (CDUs) circulate liquid through the rack to heat exchangers. Some operators are piloting immersion cooling, submerging hardware in dielectric fluid.
Liquid cooling has side benefits: it reduces fan power, allows higher chip temperatures and can improve PUE (power usage effectiveness — total facility power divided by IT power; 1.0 is perfect). Leading hyperscale facilities report PUEs close to 1.1.
The grid is the bottleneck
Building a data center takes one to two years. Building new transmission lines and generation can take far longer, and grid interconnection queues in many regions stretch for years. As a result:
- Sites are chosen for power availability first, then fiber and land.
- Operators build on-site generation — gas turbines, fuel cells and batteries — to bridge until grid power arrives.
- Utilities are revising rate structures so large loads pay for the infrastructure they require.
- Training jobs, which are flexible about location, can be placed where power is cheap and abundant; latency-sensitive inference stays closer to users.
Nuclear's comeback
Tech companies want firm, around-the-clock, low-carbon power, which has revived interest in nuclear energy. Microsoft signed an agreement to restart a reactor at the former Three Mile Island site in Pennsylvania; Amazon and Google have announced investments in small modular reactor developers; and several companies have signed long-term contracts with existing nuclear plants. New reactors take many years to build, so in the near term natural gas, renewables plus storage, and grid upgrades carry most of the load.
Custom silicon and efficiency
If power is the constraint, the key metric becomes tokens per watt. Every generation of accelerators improves it, through:
- Lower-precision arithmetic (FP8, FP4) that does more work per joule.
- More HBM bandwidth per watt, and larger scale-up domains that reduce communication overhead.
- Custom chips — Google TPUs, AWS Trainium, Microsoft Maia, Meta MTIA — designed for specific workloads and facilities.
- Software: better batching, quantization, speculative decoding and model distillation (see inference economics).
Networking at campus scale
Large training clusters span multiple buildings. They require hundreds of thousands of optical links, and the power used by optical transceivers becomes significant. That's driving co-packaged optics, which put optical engines directly next to switch chips, and new techniques for training across multiple data centers connected by long-haul fiber.
Water, communities and sustainability
Evaporative cooling can consume significant water, a concern in drought-prone regions. Closed-loop liquid cooling uses far less, and many operators now report water-usage effectiveness alongside PUE. Local communities weigh jobs and tax revenue against grid strain, noise and land use, and some jurisdictions have introduced moratoria or new requirements for large data center projects.
What to watch
- Gigawatt campus announcements — and how many actually get built on schedule.
- Utility capital plans and new generation coming online.
- Adoption of liquid cooling across colocation providers, not just hyperscalers.
- Efficiency gains from new chip generations versus growth in agentic token demand.