The transformative potential of AI is well-documented, with applications in everything from customer service to data analysis. But many CIOs are grappling with a surprisingly complex question: is it worth the money?
Nearly nine in 10 (87%) organisations are actively developing GenAI initiatives, but only 35% have a clearly defined vision for how they will create business value from GenAI, according to Bain Research.
And there are different views on just what constitutes success. Consensus on how to measure the success of AI investments is rare, according to a survey of nearly 600 CIOs and heads of IT by Gong, a sales platform.
So where are major firms focusing their AI investments – and how do they measure the impact? According to the survey, 55% focus on productivity, but a similar share look at efficiency and revenue (53% each), while 46% focus on employee satisfaction.
Taking AI’s measure
AI has transformed insurer AXA’s business, bringing benefits everywhere from customer service to risk assessment and fraud detection. By introducing a corporate version of GPT to its call centres, call resolution time has been slashed from five minutes to five seconds, as agents can swiftly retrieve policy document references to customer questions, says AXA’s UK and Ireland CIO Natasha Davydova.
AI-enabled pricing platforms have made pricing more efficient, helping the company’s underwriters complete their customer risk assessments and pricing proposals in hours rather than weeks. AI-enabled IT observability tools, meanwhile, have been implemented to detect and prevent IT incidents, reducing the total number of incidents and pushing down the time to fix.
A trial of Microsoft Copilot, which helps teams summarise and draft documents, is being extended, with its value based on productivity increases, error reduction and employee satisfaction, all of which have changed for the better, according to Davydova.
Most CIOs are trying to simplify their operations, but the issue with AI is, what do you remove off the back of it? You’re adding tech, but not taking anything away
This comprehensive tech stack doesn’t come cheap. Davydova declined to provide details of AXA’s spending, saying the company’s AI budget is confidential. However, ChatGPT at enterprise level is quoted to cost $30 (around £24) per user per month. The price for Microsoft’s Copilot Pro, meanwhile, is currently published at £19 per user per month. In a 150,000 strong global business like AXA, this would amount to £43.2 million and £34.2 million respectively.
Still, that’s not a huge outlay for a giant like AXA. The company’s total tech spend in 2023 of all platforms, not just AI, was reported by Global Data to be $2.2 billion (£1.74bn).
And the changes are already making a difference to AXA’s bottom line, Davydova says.
Indeed, the company has already recorded profit growth of 5% this year, according to industry media reports, with life and health insurance premiums up 7%, as AXA’s half year 2024 results confirm.
So how does it measure the value of the technology? Davydova believes the best use of AI is in customer-facing areas, which “help enhance the growth of our revenues and profitability, because customers choose to stay with a high-quality supplier of insurance services”.
She adds: “By analysing KPIs from operational efficiency and enhanced customer experience to improved risk management, cost savings, and employee productivity, the evidence indicates that AI contributes positively to the company’s goals and bottom line.”
Asking the right questions
Jean-Philippe Avelange is CIO at Expereo, a business connectivity company. He says all conversations around AI initiatives begin with a question: “What’s our starting point that we want AI to help with, if it can?”
Avelange says Expereo decided in 2023 to upgrade its Salesforce platform to the AI-enhanced Agentforce offering, which includes features like real-time AI-powered guidance in customer interactions. The company did not share financials, but a total package at the published price of $500 (around £394) per user per month, would amount to $2.4m (about £1.9m) for the 400 Expereo employees using the platform.
Like Davydova at AXA, Avelange emphasises customer service, with ROI metrics concentrated here. He also correlates AI platform rollouts to productivity gains and any resulting increases in employee satisfaction.
Avelange outlines key focus areas for Expereo. “How many emails are we sending per customer service agent? How long does it take an agent to handle a case summary? How much time is spent on a customer update? We then assess the cost for that specific AI use case, start prototyping and commence frequent rollouts to gather quick feedback,” says Avelange.
The hidden costs
New technology comes with a financial price, but there’s also an environmental cost that CIOs shouldn’t overlook, notes Louise Bunting, CIO at Carbon Net Neutral Technology Solutions, a corporate carbon-measurement and management company.
For example, it took 1,287MW/h of electricity to train the large language model (LLM) GPT-3, according to the Association of Data Scientists; that’s roughly equivalent to the usage of an average American household over 120 years. Gartner has predicted that by 2030, AI could consume 3.5% of the world’s electricity, while each GPT query uses about half a litre of water to cool its servers.
This all adds to an organisation’s carbon footprint, Bunting warns, which CIOs must consider when assessing the ROI of AI. “Most CIOs are trying to simplify their operations, but the issue with AI is, what do you remove off the back of it? You’re adding tech, but not taking anything away. If you’ve got a carbon target, you’re adding something which is probably the most power-hungry system, consuming up to four times more than a standard technology stack. That is a big problem from an environmental perspective,” says Bunting.
Bunting recommends a particular line of questioning when considering whether AI will be worth the cost. “Is it actually adding value? Or could you do what you need to with tech that you’ve already got? Is it actually going to save us any money, when we could do the same thing through the automation and digitisation of processes, without AI?”
Avelange says that upgrading Expereo’s existing Salesforce tool to Agentforce has helped to minimise the additional costs that come with bringing in new AI products, such as network usage spikes, plus the need for extra bandwidth and security layers.
“It’s not a simple boxed product that you buy and you’re ready to go,” he says. “Working directly in our existing Salesforce platform alleviated this risk and ensured we could keep costs contained while maintaining ROI.”
Yet another overlooked cost of AI, Bunting adds, is governance. That includes the need for specialist staff members to create frameworks and processes for how AI should be used and monitored in a business, as well as additional legal support to clean up the mess if AI gets it wrong, which is still a reality. An AI governance director can earn a salary of up to £74,000, according to Glassdoor.
“If the value of AI is that I can respond to my customer in five seconds, does that warrant having a whole team governing its use?” asks Bunting.
As Avelange notes, quick wins aren’t everything. “This risk of any organisation making short-term considerations about ROI on AI initiatives is that they could then potentially miss out on any long-term gains and benefits,” he says.
He’s convinced that AI “will profoundly transform the way any company operates” and that it “is not a debate of knowing whether AI is worth it or not”. However, he’s realistic about its complexities.
“It is a matter of survival for enterprises to adopt AI, while remaining very conscious of the costs and hype around it,” says Avelange.
