It’s difficult for the average person to contemplate how much power modern data centers are drawing. I saw similar stories to this in the past couple of months that started with references to electricity demand spikes caused by cryptomining then began referencing some of the data centers being built explicitly because of AI demands.
There are communities with data centers in their midst that have seen electric rates skyrocket because the neighboring data center (whether its operators and tenants knew it or not) was seeing 30-50 percent of its “compute” being expended on cryptomining software. It might have resulted from legit cryptominers buying the compute power above board and using it or it might stem from tenants having their existing systems infected with malware that was performing mining with stolen compute. Either way, demand goes up, the utility pays more on the spot market to meet the demand and ALL electric customers pay the higher rate.
How much power are we talking?
Here’s the mental exercise I started thinking through… Everything is based on Ohm’s law V = iR and a related equation Power = VI
Traditionally, people think of a light bulb as a small power load. It actually IS a lot of power. An incandescent 100 watt bulb plugged into a 120 VAC outlet is drawing 100/120 or 0.83 amps of current.
A typical Intel i7 CPU in your laptop or desktop computer might draw about 10-20 watts when idling or when the user is doing simple stuff like typing in a word processor. If the processor is actually busy recalculating a spreadsheet, streaming a YouTube video in a browser or playing a video game, the power draw will be closer to the CPU’s limit which is around 65 watts.
If you have a souped up dedicated gaming machine with a $700 video card providing a separate Graphics Processing Unit (GPU), that GPU card alone might draw 200 watts during heavy rendering.
So a single “home use” PC might draw up to 265 watts during heavy use.
In a data center environment, a typical server might have two CPUs, 64 or 128 GB of RAM and a bunch of hard disks or solid state drives installed pulling power. But software loads are mapped to servers to keep them as physically busy as possible. While a home PC might run at 100% of its capability for 10% of the day, a server in a data center might operate at 85% of peak power continuously, 24x7.
As an example, a 2 rack-unit Dell PowerEdge R750XA has two Intel Xeon Silver 4310 CPUs that can draw up to 120 watts each. Assume memory draws about 3 watts per 8GB (total of 48 watts for 128 GB RAM) and drives draw about 3 watts each (12 watts total for 4 drives), the server’s peak power draw could be 300 watts. If that server operates at 85% of peak 24x7, that’s a draw of 300 x 0.85 = 255 watts per server.
One standard rack in a data center provides 48 “rack units” of vertical space so one rack can fit 24 of these 2RU sized servers, so a single rack might draw 255 x 24 or 6.12 kilowatts of power per rack. With 120 VAC power, that requires delivery of 51 amps to each rack.
If those servers are being installed to process AI data, each server will also include GPU cards like a home PC. They won’t be used for driving displays but the memory and processor threads used for graphics rendering are equally valuable in crunching the matrix mathematics associated with creating and using large language models. At a minimum, that adds another 200 watts of power draw per server, bringing that draw up to 500 watts, again being utilized at least 85% of the day each day, 24x7. That’s 500 x 0.85 x 24 = 10.2 kilowatts per rack.
So how much processing power is being added to meet demands for Artificial Intelligence uses? Microsoft alone is planning to build a campus of data centers to house at least one million “AI chips”, likely a reference to a single GPU card.
If Microsoft’s server design fits two GPUs per 2RU server, power consumption goes up by another 200 watts per server to 700 resulting in 700 x 0.85 x 24 = 14.28 kilowatts per rack. If Microsoft is building space for 1,000,000 “AI chips”, that’s
servers = 1,000,000 GPUs / 2 GPUs per server = 500,000 servers
which is
racks = 500,000 / 24 servers per rack = 20,833 racks
which is
power = 20,833 x 14.28 kilowatts/rack = 297,500 kilowatts
Over the course of a 30-day month, that’s
monthly AI power = 297,500 x 24 hours x 30 = 214,200,000 kilowatt hours
For comparison, you can look at your own electric bill and calculate how many equivalent “yous” there are in that total. My peak electric bill occurs in August of each year and amounts to 540 kilowatt hours in the month costing $93. The monthly power draw of this planned Microsoft data center is equivalent to
Equiv households = 214,200,200 / 540
= 396,667 individual households
Assuming I haven’t mis-converted units between watts, kilowatts and kilowatt hours somewhere, that is a staggering multiple. And that power draw is CONTINUOUS throughout the year, not seasonal just for a couple of months.
As an additional caveat, the type of GPU likely to be purchased for core AI functions is not a typical $700 gamer GPU or even the more expensive $1800 deluxe GPUs used by fanatical gamers. Nvidia makes a different line of GPUs, an example being their A100 GPU, that costs nearly $10,000 per unit and draws 400 watts peak per card. These require installation in sets of 4 or 8 using a special interconnect board provided by Nvidia which fits into a server chassis that is typically a 4RU height chassis so if the A100 is used in this new data center,
- one server = 8 GPUs
- power per server = 300 + 8 x 400 = 3500 watts
- servers per rack = 48 / 4 = 12
- power per rack = 12 x 3500 = 42,000 watts
- GPUs per rack = 96
- racks for 1,000,000 GPUs = 1,000,000 / 96 = 10,416 racks
- total power = 10,416 racks x 42,000 watts/rack = 437,500 kilowatts
As these numbers illustrate, it’s a pretty easy case to argue that any advances we are making in improving energy efficiency to combat global warming are being swamped by additional electricity demand for AI.
WTH
