Despite $30–40 billion in enterprise investment into GenAI, this report uncovers a surprising
result in that 95% of organizations are getting zero return.
https://www.rand.org/pubs/research_reports/RRA2680-1.h
tml
By some estimates, more than 80 percent of AI projects fail — twice the rate of failure for information technology projects that do not involve AI.
https://www.techrepublic.com/article/why-85-of-ai-projects-fail/
Why 85% of AI projects fail
What gives?
It takes a while to figure out how to use new tech effectively. ChatGBT was only publicly released 3 years ago. There are projections that AI will increase global GDP growth by 1% per year. That should show up on a chart, but no signs of it yet. World GDP annual growth:
Surprising to anyone not familiar with the Pareto or Power Law Distribution.
The Captain
Does that apply to other technologies like telephones or computers?
To just about everything. Pareto discovered that in Italy 20% of people owned 80% of the land. Then others certified it was true elsewhere.
My first encounter with the Pareto distribution was in 1964 when IBM was peddling IMPACT (Inventory Management Program and Control Techniques). IBM did not mention Pareto by name and I only found out about his decades later.
IBM has investigated these new methods and has developed a standardized approach termed “IMPACT” - Inventory Management Program And Control Technique. To assist distribution industries in implementing positive control over when and how much to buy, an IMPACT Computer Program Library has been developed. This manual presents the principles of IMPACT in layman’s language.
See also
The Captain
o o o o o o o o o o o o o
IMPACT was one of the reasons IBM and I parted company. IBM sales reps made a lot more money that system analysts (me) and when I asked for a raise they told me if I wanted that kind of money I had to become a sales rep. I agreed. They sent me to Sales School in Cuernavaca, Mexico and gave me a mostly virgin territory to develop.
Prospects in Barcelona (Vzla) said they would buy IBM if there was a local office. At IBM they said they would open the Barcelona office if Barcelona bought IBM equipment. Catch-22! One day I was asked to organize an IMPACT conference which seemed an excellent way to break the impasse. I visited the president of an important sugar mill to propose the idea. He loved it and volunteered to host it. Then the bad news broke out, the IMPACT EXPERT from the USA could not give the talk! So I suggested I give the talk which was denied. Some time later my local manager asked me to give the IMPACT talk to an important customer in Valencia (Vzla). I did and asked him how it went. Good. “Then why the fork did you people not let me give the talk in Barcelona to open that territory?”
Managers from IBM World Trade made me an offer to become the sales rep of Trinidad and If I made my quota for two years I would be appointed the Trinidad IBM General Manager. After the IMPACT incident I said to them, “I’m not liking this sales business.” Talk about a fork in the road!
Soon after IBM and I parted ways.
o o o o o o o o o o o o o
Google AI Overview
Yes, Italian economist Vilfredo Pareto observed in 1896 that approximately 80% of the land in Italy was owned by 20% of the population. He later found similar distributions in other countries, and the concept was later expanded upon by management consultant Joseph M. Juran. This principle, now known as the Pareto Principle or the 80/20 rule, suggests that a minority of causes or inputs produce a majority of outcomes or outputs.
Pareto’s initial observation: Pareto was studying wealth distribution in Italy and noticed that about 80% of the land was owned by 20% of the people.
Expansion to other countries: He then surveyed other countries and found the same pattern applied to wealth and land ownership, says Wikipedia.
“80/20 Rule” popularization: Management consultant Joseph M. Juran popularized the principle in the 1940s, applying it to business and quality control, according to Anexas and The Decision Lab. He referred to it as the principle of the “vital few and trivial many”.
Broader applications: The 80/20 rule has since been applied to many other areas, such as the idea that 80% of a company’s revenue often comes from 20% of its customers.
Not that this has anything to do with investing, but I had a 595 rule at work. 5% of my employees caused 95% of my problems. Fortunately, 95% of my employees were great.
GCR2016,
That applies to machines, processes, weather, measurements, and raw materials, too!
The below diagram is an Ishikawa diagram which helps groups of people identify and qualitatively diagnose problems.
If you don’t have data to collect, Cause and Effect!
The issue is why is the AI failure rate so much higher than the non-AI failure rate. Saying “Pareto” doesn’t address that question.
I contend that the reason is because AI is turning out to be less powerful than we thought. Here is one example for why this might be. Suppose the objective is to cure Alzheimer’s. You can have one AI with its single set of optimized algorithms analyze all the data and come up with a plan of action. Or you can have a thousand human scientists with each brain representing a different set of algorithms look at the data from a 1000 different perspectives. Which approach is likely to be the most innovative and creative?
I think it is the latter.
I think AI is potentially very good at incrementally optimizing what exists. But when it comes to developing new ways to do things it is difficult to beat the system of many smart humans competing for money and recognition.
I don’t know about specific failure rates but the observation is valid.
It’s not that AI is less powerful but that expectations are overinflated as per the Gartner Hype Cycle. Five billion humans vs. a handful of data centers. That needs to change. Elon Musk commented about their newer inference chip which apparently will be able to do some training, or so I understood. If true the ratio would change to five billion humans vs. a few million Tesla cars, Optimus robots, and other devices using the chip.
Please don’t use the term “algorithms,” there are no algorithms, there are probability gradients. What exists is “pattern matching.” In layman’s terms “Let me check what I see against the probability tables I have to decide what I should output.” Richard Feynman uttered a very appropriate phrase in one of his lectures, “We start with a guess, all it is, is a guess.” Call it theory, hypothesis, or some other fancy name but what it really is, is an EDUCATED guess. Then the Scientific Method kicks in to falsify or validate the guess.
My search for IBM’s IMPACT revealed one of the shortcomings of the current capabilities of Large Language Models. When I searched for:
when IBM was peddling IMPACT (Inventory Management Program and Control Techniques)
Google AI was practically offended, “IBM did not peddle Inventory Management…” and it recited what IBM had been doing in the last decade. IMPACT dates back to 1962! No Internet, no text to train AI, proving George Santayana right about ignoring history! To find the pdf I linked I had to search multiple times to get around Google AI’s ignorance, a term properly used. Ignorance of history (1962) and ignorance of literary irony (peddling).
While currently true I believe in time AI will develop the ability to infer based on the model of complex systems. When AI can’t come up with the right answer to a question it might guess something related as it did with my IMPACT query. Like Richard Feynman said, “It starts with a guess, an educated guess…” AI is not yet educated about 1962 software, it had to do with only newer data.
Thanks for a great conversation starter!
The Captain
Please don’t tell that to the people who actually work in the industry. They call them algorithms. For a reason. As just ONE EXAMPLE OF MANY look up the back propagation algorithm.
How long did it take for the average person to use personal computers in the early 1980’s. I think we had two for a department of 50 people when I worked for Exxon in Houston.
intercst
Exactly (and thank you).
Hand-wringing over the word algorithm is nonsense.
Any collection of formulas (which certainly can include “algorithms”) with parameters that makes a prediction (inference) can be trained in the machine learning sense.
Neural networks are a kind of collection of formulas.
Yes, the code they write are algorithms but AI does not write algorithms, it mimics how the brain works. I believe the distinction is important to understand AI.
To be more precise, when we write traditional algorithms it is to make the computer act like a boolean brain (AND, OR, NOT, etc.). Neural network AI has two modes, training and inference.
The training part has two phases. Algorithms create the neural networks. The neural networks are then fine tuned by adjusting the weights of the nodes.
The inference part does the pattern matching (no algorithms) which should be followed by validation of the “guesses” it comes up with.
The above is how I perceive AI architecture. YMMV
The Captain
And “the internet is a series of tubes” – (Sen Ted Stevens (R. Alaska))
intercst
Well, I’m no “tubes” expert, but just so there is no confusion, I don’t consider those to be similar statements of understanding.
The former statement is accurate (I’ll let someone else explain the internet as tubes).
Backpropagation algorithm, by the way, is just calculating a derivative on a neural network (because, you know, formulas and such).
It is important. And you keep getting it wrong.
The Captain
