Why AI Needs So Much Power: Data Centers, Energy, and the Cost of Intelligence

A single AI rack now draws as much power as a small neighborhood. One chat prompt costs almost nothing. A billion prompts a day, plus the training runs behind them, plus liquid-cooled racks packed edge to edge, add up to a load that grid operators now plan around. This guide traces the power from the wall socket to the token, with real numbers at each step.

Quick Takeaways

• Global data center demand: The IEA projects consumption will grow from 415 TWh in 2024 to 945 TWh per year by 2030.
• US share: A June 2026 Berkeley Lab analysis estimates data centers could reach 9.5 to 15 percent of US electricity use by decade’s end, up from about 4.4 percent in 2023.
• Per prompt vs. at scale: A median Gemini text prompt uses 0.24 Wh. Reasoning models and agents use far more, and total demand is driven by volume.
• Density is the bottleneck: A GB200 NVL72 rack draws roughly 120 kW, so power delivery and cooling limit how fast capacity can be built.

Where the Electricity Actually Goes

A data center is a machine that converts electricity into heat and computation, in that order. The split matters because each slice has a different fix.

IT load is the servers themselves. The IEA expects AI-optimized servers to account for almost half of the net increase in data center consumption by 2030.

Cooling is the next biggest slice. Cooling and humidity control account for around 20 percent of a data center’s total consumption.

Power conversion and distribution lose energy at every step: grid to transformer, UPS, power shelf, and finally the board’s voltage regulators.

Power Usage Effectiveness (PUE) captures the overhead. It is total facility power divided by IT power. The IEA expects the global weighted average to improve from 1.41 to 1.29 by 2030. A PUE of 1.3 means 30 extra watts of overhead for every 100 watts of compute.

Why AI Is Different From Classic Cloud Computing

Web servers and databases spend much of their time idle. AI accelerators do not. Three properties drive the gap.

1. Dense, parallel math at high clock utilization

Training and serving a transformer is dominated by large matrix multiplications. GPUs and TPUs run these at very high utilization, so they sit near their thermal design power (TDP) for long stretches. A classic CPU server idles at a fraction of peak.

2. Memory movement costs energy

Autoregressive decoding generates one token at a time. Each step reads the model weights and the KV cache from HBM. Moving bytes often costs more energy than the arithmetic itself. This is why quantization (FP16 → FP8 → FP4) and batching save real power.

3. Rack density has jumped

Older facilities were built around about 20 kW per rack. The GB200 NVL72 packs 72 Blackwell GPUs into one liquid-cooled rack. HPE advises provisioning for a peak electrical design power of 1.5x the 132 kW nominal TDP, about 192 kW.

That 120 kW rack is roughly 3x an air-cooled rack of H100s. A building cannot simply add racks. As one analysis notes, a facility that can power one rack cannot assume it can power two without re-running the electrical math.

Training vs. Inference: Two Different Power Problems

Training is a one-off, enormous, concentrated load. A frontier run keeps tens of thousands of accelerators synchronized for weeks. The electricity bill is paid up front, and the cluster needs a dedicated grid interconnect.

Inference is a continuous, distributed load. Each request is small. Millions of requests per minute, around the clock, make it the larger share of lifetime energy for popular models.

Per-prompt energy depends heavily on what the model does:

Google reports its median prompt energy fell 33x in one year through software efficiency and clean energy procurement. The caveat: per-prompt numbers can normalize a growing footprint if total query volumes explode. That is the Jevons effect in practice. Cheaper tokens invite more tokens, and agents that call a model dozens of times per task multiply the load.

Back-of-the-Envelope: Power for a GPU Cluster

Use this script to size a cluster’s grid demand. Swap in your own rack power, PUE, and utilization.

# Cluster power estimator — all inputs are assumptions you can change
GPUS = 100_000          # accelerators in the cluster
GPUS_PER_RACK = 72      # GB200 NVL72
RACK_KW = 132           # nominal TDP per rack (kW)
PUE = 1.3               # facility overhead multiplier
UTILIZATION = 0.8       # average fraction of nominal power drawn
HOURS_PER_YEAR = 8760

racks = GPUS / GPUS_PER_RACK
it_mw = racks * RACK_KW / 1000
facility_mw = it_mw * PUE
annual_twh = facility_mw * UTILIZATION * HOURS_PER_YEAR / 1_000_000

print(f"Racks: {racks:,.0f}")
print(f"IT load: {it_mw:,.0f} MW")
print(f"Facility load (PUE {PUE}): {facility_mw:,.0f} MW")
print(f"Annual energy at {UTILIZATION:.0%} utilization: {annual_twh:.2f} TWh")

At full load, a 100,000-GPU cluster needs about 238 MW at the meter. That is roughly 2.1 TWh per year at 100% draw. At 80% utilization the script prints about 1.67 TWh.

For serving, the same logic applies per request:

queries_per_day = 1_000_000_000
wh_per_query = 0.24      # Google's disclosed median
mwh_per_day = queries_per_day * wh_per_query / 1_000_000
print(f"{mwh_per_day:,.0f} MWh/day")   # 240 MWh/day, ~10 MW average

The Grid Problem: Speed, Not Just Size

The IEA says data centers will account for around 10 percent of global electricity demand growth by 2030, less than industrial motors, air conditioning, or EVs. Globally, the load is manageable.

The trouble is concentration and speed. In the US, the IEA expects data centers to drive almost half of the growth in electricity demand through 2030. Demand lands in a few regions, such as northern Virginia, Texas, and Ohio. Substations and transmission lines take years to build, and a data hall takes months.

The IEA warns that without transmission investment, up to 20 percent of planned data center projects could be at risk of delays. Its April 2026 update says data center demand rose 17 percent in 2025, more than five times faster than overall demand growth. Developers increasingly turn to on-site generation and gas-powered systems, though many of those projects are early stage.

The generation mix matters for emissions. 46% of data center electricity in 2024 came from fossil fuels, mostly gas in the US and coal in China. The IEA projects renewables will supply an additional 450 TWh by 2035 for this demand, with gas and nuclear each adding around 175 TWh by 2030.

Levers That Cut the Bill

Efficiency gains come from every layer of the stack.

Liquid cooling is a clear example. Vendors report next-generation AWS data centers with liquid cooling increase compute power by 12% while cutting energy consumption by up to 46%. Treat this as a vendor claim for a specific design, not a universal figure.

Real-World Scenarios

Enterprise: choosing the right model size

A support team routes 10 million tickets a month through an LLM. Most tickets are simple. A routing layer sends easy cases to a small distilled model and escalates hard ones to a frontier model. Energy and cost both fall, and quality holds where it matters.

def route(ticket_complexity: float) -> str:
    # Threshold tuned on a labeled evaluation set
    return "small-8b-fp8" if ticket_complexity < 0.6 else "frontier-reasoning"

Developer: cap reasoning effort

Reasoning tokens drive the high end of the energy range. Set a max thinking budget for tasks that do not need deep deliberation, and cache repeated system prompts. Fewer generated tokens means less compute and lower latency.

Utility planner: treat AI load as flexible

Training runs can pause or slow for hours. Grid operators and developers are exploring flexible interconnection, where a campus accepts curtailment in return for faster grid access. Inference for real-time products is stiffer, but batch inference, embeddings, and evaluations can follow the grid.

What the Numbers Do and Don’t Show

Per-query figures are not comparable across vendors. Google reports a median and OpenAI’s figure is described as an average, so the two can’t be compared directly. Google’s number also excludes training and hardware manufacturing. Forecasts spread widely: LBNL’s 2024 report gave 6.7 to 12% of US electricity by 2028, and its 2026 update gives 9.5–15% by 2030. The range reflects real uncertainty about chip shipments, utilization, and efficiency gains. Plan around ranges, not point estimates.

FAQ

How much electricity does one AI prompt use?

A median Gemini text prompt uses 0.24 Wh, about a second of microwave time. Reasoning-heavy requests can use many times more, with independent estimates running from roughly 4 to 40 Wh. Image and video generation typically cost more than text.

Why do AI data centers need so much power?

Accelerators run dense matrix math near peak power for long periods, and moving weights and KV cache through memory adds to the draw. Racks now reach 120–132 kW, and cooling plus power conversion add overhead on top. Millions of daily requests then multiply the load.

Will AI consume most of the world’s electricity?

No. The IEA’s base case has data centers at around 3% of global electricity by 2030 and about 10% of demand growth. The issue is local: a few grid regions absorb a large share of new load, and interconnection takes longer than construction.

Can AI become more energy-efficient?

Yes, and it already is per prompt. Google reports a 33x drop in median prompt energy in a year. Total demand can still rise if usage grows faster than efficiency, which is why forecasts keep climbing.