Will AI's power draw doom Nvidia (and by extension others)?

Not buying that this is enough to kill off AI but … an interesting question

Huang says Moore’s Law is dead because Nvidia’s price-performance is increasing well ahead of what Moore’s Law could predict.

To accomplish this, however, Nvidia is rewriting rules around chip design. A single Blackwell chip will draw 1,875 watts of power. That’s nearly twice what the current Hopper chips consume.

That means 1 million Blackwell chips, which Nvidia plans to produce, will be drawing twice the power of a nuclear power plant.

The capability of Nvidia hardware is also off the charts. A single server rack equipped with Blackwell chips will deliver 1.44 ExaFlops of performance. An ExaFlop is a quintillion, 1 followed by 18 zeros.

That means a single server rack can do the work of an entire data center. In practical terms, it means the clouds are obsolete. Anyone can have one. When Elon Musk diverted a $500 million Nvidia order from Tesla (NASDAQ:TSLA) to his X service, he was grabbing a cloud data center.

The Bottom Line

That giant sucking sound you hear is the world’s wealth, and the world’s energy, flowing into Nvidia data centers.

It’s too much.

The last decade’s cloud systems were built with energy consumption in mind. Alphabet (NASDAQ:GOOG)(NASDAQ:GOOGL) made it a practice to recycle the energy it used for things like water treatment. Microsoft and Amazon planned renewable energy projects around their cloud energy needs.

Nvidia is blowing all this up.

Nvidia is also well ahead of what software can do. Funding for generative AI startups is starting to dry up. Even the Cloud Czars are unclear how they’ll get their money out from AI.

The current mania, like all such mania, will end. I wish I could tell you when that will happen. I can’t. It probably won’t be this quarter or even the next.

But you might want to lighten up on your Nvidia holdings before it happens. The company and the AI industry are on an unsustainable path.

2 Likes

We need those AIs to get cracking on solving the problem of sustainable nuclear fusion reactors NOW!

3 Likes

I’m sure the LLMs will happily describe a fictional version of a working Fusion reactor any time now :confused: “Hi, I’m hallucinating about how you could continue to feed me energy.”

1 Like

Not really. It means that Blackwell int8 (I assume) convolution operations per second can do (roughly) the same number of int32/64/FP32 operations per second in a data center.
Neural net convolution operations are limited to combined multiply-add operations (maybe a couple of others).

Still impressive, but not a proper replacement.

Mike

Still impressive, but not a proper replacement .

Of course not. Hyperbole. But do we think there will be a throttling of the progress here related to power consumption?

Looking for energy plays that are likely to be driven by this.

1 Like

In the short term…certainly not. But we can’t just keep increasing the power consumption of a single chip (package actually). Just like the clock speed increases that happened for decades then it abruptly ended with lower speeds and multi-cores being more efficient.
Remember the 8086/88 was ~5 Mhz, 386 was 25 Mhz, Pentium 1 Ghz, Pentium 4 ~3 Ghz, then suddenly no real gains after that, even lower speeds for a more balanced multicore performance.

Mike

Mike,
It’s possible my interpretation is wrong but I think a flop for Nvidia is a 4 bit float operation. Seems totally unusable (as a float) but there you go.
Boosts their numbers though, you could do 16 of them at once instead of a double float.

There are several new data types for doing fast convolutions useful in AI…such as FP16, int8, FP8… as well as block versions of these only executable in the TPU. I don’t know about FP4, but could be.

Mike