The cost of AI Failure

Having a degree may or may not help one be a capable engineer.

Other than the quotes above, no one said anything about the relevance of a degree.

Many are self-taught in many areas and actually work in areas different from their educational training.

You may remember that during the pandemic there was an unprecedented rush to hire software engineers and others in related fields, as companies thought “Wow, everybody’s going to work from home! That means they’re going to use all sorts of new softwares and buy food and stuff online and do all sorts of golly-gee whiz bang things.” I recall stories of people getting signing bonuses of stock and grants in the millions!

There was a FOMO of companies hiring because everybody else was hiring and they were afraid all the “good ones” would be snapped up, so they hired in spite of the fact that they had no business case for them.

I’m here to say they’re getting rid of them because … there was no business case for them, and none has developed. Could AI be playing a part? Sure, but it’s hard to tell the wheat from the chaff when a mania like that which happened a few years ago occurs.

Speaking of which: Data centers, AI chips, gas turbines, and all those related rocket-ships that we’re enjoying. At some point one or two of them is going to look around and say “You know, we might have enough (too much) of this” and then won’t it be fun?

6 Likes

My supply of rocket ships is adequate.

Same for AI chips.

2 Likes

That is, I think, the very real possibility. One of the interesting things about the LLM AI models is that they all seem to have converged very closely into having the same capabilities and features. Sure, there’s small differences. But Grok and Claude and Perplexity and Gemini and ChatGPT and CoPilot and the rest have all ended up being very similar.

We almost certainly don’t need 6-8 different versions of essentially the same fungible product, which makes it very unlikely that we’re going to keep running 6-8 different versions of the same fungible product. The companies involved kind of know this, which explains some of the early efforts to kind of distinguish them from each other with IP or gimmicks (“Sora has Disney characters!” “Grok has sexy pron-y features!”). They’ve kind of walked that back in recognition that those things aren’t likely to work well and because they think Anthropic found the money line: these things will make money first on the enterprise side, not the consumer-facing side. But the same dynamic will play out there: we don’t need to support 6-8 different competing versions of an AI coding model, if they all do pretty much the same stuff the same way.

There will be a winnowing. You have too many companies racing to build out the largest AI God-Thingies - more than we need to have. Which means that some of those AI God-Thingies will be superfluous.

3 Likes

I have used Claude, Chat, and Gemini; that is not true. It is as if there are levers to slide up and down for the results between detailed responses and general purpose responses. They each take a different tactic, which means they each specialize differently.

Which means that capacity will be “available”, and the pace of this new gold rush will slow, if not stop, because it’ll be easier/faster/cheaper to just rent from the other guy than to build it all from scratch yourself.

Nah!

courtesy of Google

The cost of AI computing is falling at an exponential, industry-redefining rate, with inference costs for some models dropping by over 99% (or 280x to 900x) between late 2022 and early 2026. This dramatic deflation, often outpacing Moore’s Law by over seven times, means high-end AI capabilities are shifting from expensive, specialized tools to affordable, commoditized utilities.

[image]Medium·Emma Kirsten +3

Key details on the accelerating decline of AI costs include:

  • Inference Price Plunge: The cost of running AI models (inference) has dropped significantly, with some benchmarks showing reductions of 280-fold over two years, moving from over $20 per million tokens to under $0.10 in some cases.
  • Hardware Efficiency: Hardware price-performance improves roughly 30% to 40% annually. New chips, such as NVIDIA’s Blackwell, offer a 30-fold improvement in speed over predecessors, significantly reducing costs-per-performance.
  • Rapid Efficiency Gains: Researchers suggest the cost to perform AI tasks is halving every 2 months.
  • Model Optimization: Improvements in software and algorithmic efficiency, such as techniques used by DeepSeek, have reduced inference costs by 97% recently, while smaller, optimized models are rivaling the performance of much larger ones.
  • Future Projections: OpenAI CEO Sam Altman has stated that the cost of using AI could drop by 10 times every year.
  • Total Spend Rise: Despite the falling cost per unit, total expenditure on AI continues to rise because usage is increasing even faster than costs are falling, resulting in massive, high-throughput enterprise bills.

[image]Harding Loevner +8

This rapid deflation makes AI applications, which were previously economically unviable for businesses, suddenly feasible, with many tasks shifting from being “too costly” to highly profitable.

[image]Harding Loevner +1

1 Like

Grok is out. Perplexity is taking a different path, they are/ were not in the LLM business. There are differences between Claude, Gemini, and ChatGPT. They may all survive and thrive.

AI is evolving so fast, talking about difference is meaningless, because no one knows what others have in their sleeve.

It’s not meaningless any more than talking about the differences and potentials for Google, Excite, Yahoo!, AskJeeves, and AltaVista were in 2000. At some point a meaningful differences appeared, and one was a runaway while the others crashed and burned.

Now this will likely be different because “search engines” were entirely consumer facing, while these iterations of AI will (purport) to serve different markets. Facebook’s, for instance, can be a Wholly-Owned-Subidiary® and be OK *(maybe). They will claim it optimizes targets for advertising and makes “summaries or comments” more useful, and so on. They can rationalize it to any degree they want, so long as it is entirely internal and their revenues keep rising.

Google might do the same, it’s an easy rationalization. Microsoft, a bit harder, but rising income masks a lot of sins. The others, groping for sustainable markets, may find some, but perhaps not to the degree that justifies the astonishing spend (or maybe it will, yes, this is an unknown). A couple are bound to shake out, and I suspect that a dip in the economy will cause at least a couple to fold.

I don’t think this is going to be a “winner take all” field, simply because AI may be useful for many things, both for general consumers, for companies’ pricing and logistics, for coding, for media, an d so on - and I suspect we will see them playing to whatever strengths they develop as time goes along. Right now it’s all hazy, for sure.

I am reminded of articles I read in 1998 wondering which on-line seller would do better: Amazon or eBay. For a while the money was on eBay, as it exploded and decimated newsprint classified ads overnight and was profitable almost immediately. Meanwhile Amazon was plodding along, selling books, and then a few other things and losing money hand over fist. In the case of these AI wannabes, there’s no telling which markets will open, which will seem good but falter, and which companies will follow which paths to glory. Or not.

3 Likes

Google was actually found on 1998. Just saying… I think we are still at 1995…

Again, Web server, the engine behind web was actually introduced in 1990, and by 1997 web logic server had introduced mid-tier…

The apache HTTP server was introduced in 1995..

Those who are familiar with Old berkshire board, there were pundits, thundering IBM’s websphere alone worth $100 B… today is the day I will forever forgive them…

The reason I am mentioning this is, the web was not created as consumer facing tech, rather enterprise tech. Those who are familiar with LAMP stack.. if they can travel back in memory… will realize the good old days…

1 Like

That seems off. ChatGPT was launched in 2022 - it’s been around for nearly four years now.

It only took about five and a half years for the internet to go from the release of Netscape (November 1994) to the dot-com bubble popping (March 2000). It took slightly less time from the launch of Yahoo search to Google becoming the largest search engine.

I’m not suggesting that this is a mature industry by any means. But we’re not in 1995 - the absolute beginning of the First Browser War and the rise of search engines. We’re already several years into this specific product period of LLM’s and AI chatbots (AI existed well before this, of course, back when we called it “Machine Learning”). We’re not anywhere close to the end…but 3.5 years after the release of ChatGPT, we’re not at the beginning anymore either.

1 Like

Fair enough… if you compare ChatGPT launch with Netscape… I totally subscribe to that view.

I think they’re pretty different products, of course. But if we’re trying to judge how far we are into this product cycle, it’s helpful to remember that we’ve had Chat-GPT for a while now. I’ll admit that it wasn’t something I fully realized, even as I started writing that post - I was taken aback when I looked it up to find that Chat-GPT dates back to 2022. Or that the original GPT by OpenAI is eight years old now, released back in June 2018.

The big three of current LLM Chatbots (ChatGPT, Claude, Gemini) had all been released by March 2023 - more than three years ago. We’re probably in “end of the beginning” territory.

I meant the launch of chatGPT is the start of this AI cycle, similar to launch of Netscape for the internet era.

There is an uncanny relationship, including stock market cycle

1 Like

Probably to include “irrational exuberance” and “overbuild”, probably to include “eventual correction”

3 Likes

And circling back to layoffs, hiring freezes and other “management tools”…

I think there are a mixture of reasons for layoff and hiring freezes (as usual, management leverages “excuses”, “reasons” and “strategic actions” to answer the eternal question:

Why did you guys do so little efficiency/effectiveness this quarter?

At one of my former .corp enterprises, the management team was exceptionally reactive and commonly would go M&A to buy their way out of trouble. With a deep treasury and small acquisition costs, their order of operations was:

  1. Benefit from whatever tailwind was at hand
  2. ???
  3. See worsening performance
  4. Report worsening performance
  5. Announce M&A during the call (or divestitures, etc.)
  6. See worsening performance
  7. ???
  8. Layoffs

That example is akin to the “try anything until we run out of wrong answers”.

For AI, companies are doing the opposite.

  1. Accept new strategic goals
  2. Identify “Advanced Analytics” as an enabler
  3. Layoff people
  4. Blunder through while stating that “teams must be AND culture not OR culture”
  5. Voluntell functional managers that they need to find 8-10% performance/efficiency gains despite historical run rates that barely keep up with inflation through creative math.
  6. Let time take it’s toll
  7. blunder
  8. People gain efficiency because they have no other option (using AI or other process enhancements)

I am seeing this as my current .corp. I have just completed 23 months of “innovation without additional staff and budgets”. I have no expectation that I’ll get extra support in the near future. But AI tools and other “Digital Enhancements” are saving us Millions of dollars annually.

If I had my way, I’d have about 400% of my budget and 800% of my staff and I’d be saving 5x that amount.

3 Likes

Another thing, many companies insist on finding bottom 5%. It just doesn’t matter how well your team is performing. You need to identify and let go bottom 5%.

@jaagu

the video I wanted is not here, but the info on the space race for DC is here.

Absolutely! I suspect that communities dealing with the pollution of a Big Oil or Big Coal Superfund site wish they’d done one.

intercst

4 Likes

Of course there are costs. The benefits of fossil fuels have been enormous. For example, the Industrial Revolution and all of its benefits.

DB2