I’ve been asked to provide some additional information to those folks who might be interested in the facts around Artificial Intelligence (AI) and Machine Learning (ML).
I’ve provided a number of links below to IT / Business experts findings / recommendations re: current thinking on ROI of AI / ML.
The overall theme of the IT / Business experts who either authored these articles or those that were interviewed in the articles, is that most projects that are using AI / ML are not defining / tracking all of the benefits (and costs) associated with the total cost / return of these projects. Hence the ROI is either unknown, understated, or in some cases overstated - but most are inaccurate - and that’s the REAL problem.
The first article is from a couple of authors from a consulting company called Slalom. I did a one year contract with them on a data science project for one of their key Boston clients. I don’t know the authors personally, but do know that Slalom does a lot of business in the Data Science / AI / ML areas and has my confidence that this article is not BS.
https://www.slalom.com/insight/roi-ai-measure-value-deliver-…
When it comes to artificial intelligence (AI), where discovery and experimentation are central to the development of any use case worth its salt, concerns of ROI play a central role in determining where organizations should invest their time and resources. ROI can frequently be harder to calculate for data science use cases, given the widespread and sometimes nebulous nature of impacts. This makes it very important to leverage an ROI analysis at the earliest opportunity to achieve clarity in how best to prioritize high-value use cases and products.??It’s important to understand that calculating ROI is not an all-or-nothing approach. Some use cases can be justified by simply looking at obvious efficiencies gained, while others require a more robust business case to justify an investment.
This next article is dated 2019, so is a little more dated, but also describes the challenges associated with calculating ROI for AI / ML projects. I don’t think you can call Accenture a slouch when it comes to advising upper management on how best to justify IT investments. I worked with them for a number of years at a number of global utilities delivering state-of-the-art automated metering and automated customer systems.
https://www.accenture.com/us-en/insights/artificial-intellig…
Some AI applications link neatly to projected returns, making ROI calculations straightforward. An energy producer, for example, could tie its investment in an AI-powered predictive maintenance tool directly to increases in equipment uptime or reductions in maintenance costs.
Other applications are more complex and unpredictable, making it challenging to use typical ROI approaches. To what extent, for instance, could reductions in crime be tied to AI projects when many other factors may also be having an impact. Yet in any scenario, we need to make a solid business case for AI investment.
Where it is difficult to make such a business case—be it because of inherent complexity or available capabilities—organizations can risk either losing competitive advantage by delaying investments or sinking money into the wrong AI initiatives.
MIT does a fair amount of research in this area as well. Here’s a link to a recent article also describing the challenges associated with implementing and tracking ROI for AI / ML projects. I’d say MIT is a pretty reliable source of information as well.
https://sloanreview.mit.edu/article/achieving-return-on-ai-p…
Companies embarking on AI and data science initiatives in the current economy should strive for a level of economic return higher than those achieved by many companies in the early days of enterprise AI. Several surveys suggest a low level of returns thus far, in part because many AI systems were never deployed: A 2021 IBM survey, for instance, found that only 21% of 5,501 companies said they had “deployed AI across the business,” while the remainder said they are exploring AI, developing proofs of concept, or using pre-built AI applications.
…
But other companies have achieved economic return on their AI investments. Their strategies for finding value include establishing close relationships between the data group and interested business units, selecting projects with tangible value and a clear path to production, lining up trust from key stakeholders in advance of development, building reusable AI products, selectively employing “proof of concept” projects, and establishing a management pipeline or funnel leading projects toward production implementation.
So, to make a blanket statement like “The AI tech wave has had the lowest rate of return of any tech wave so far.” is totally inaccurate given the facts that most AI / ML projects struggle with, or don’t try, or don’t know how best to measure the ROI on the projects due to their complexity, unknown total costs and quantifying the savings on “soft” benefits.
I hope that these facts help folks realize that AI / ML is just like any other NEW technology. The costs / benefits are difficult to quantify at first, so these projects MAY appear to have a terrible ROI, but eventually, the hard and soft benefits and costs will be identified and an appropriate ROI calculated.
IMO, those organizations that DO journey down this path, and get good at it - in their customer engagement segment, operational areas, or even in middle management functions - will absolutely have a competitive advantage over those that do not deploy it in their businesses.
'38Packard