--- title: "AI performance: how to measure the success of artificial intelligence initiatives" description: Artificial intelligence can influence competitiveness, but it is crucial that each organization can measure its actual performance. image: https://newsroom.kireygroup.com/hubfs/AI_Investments.webp --- [kireygroup.com](https://www.kireygroup.com/) [![](https://newsroom.kireygroup.com/hubfs/item-2020-dicembre/logo-Blog-v2.svg)](https://newsroom.kireygroup.com/en/news) - [Ita](https://newsroom.kireygroup.com/news/ai-performance-come-misurare-il-successo-delle-iniziative-di-intelligenza-artificiale?hsLang=it) - Eng - [Esp](https://newsroom.kireygroup.com/es/news?hsLang=en) [Kireygroup.com](https://www.kireygroup.com/) # Get your daily dose of tech! ## We Shape Your Knowledge # [Legacy modernization in banking: discover the true cost of legacy systems](https://newsroom.kireygroup.com/en/news/legacy-modernization-in-banking-discover-the-true-cost-of-legacy-systems?hsLang=en) The banking sector is one of the world’s largest investors in digital transformation and innovation,... [Read more](https://newsroom.kireygroup.com/en/news/legacy-modernization-in-banking-discover-the-true-cost-of-legacy-systems?hsLang=en) 10 MIN READ 22 September 2026 ### Latest News - [Legacy modernization in banking: discover the true cost of legacy systems](https://newsroom.kireygroup.com/en/news/legacy-modernization-in-banking-discover-the-true-cost-of-legacy-systems?hsLang=en) - [Context Engineering, the art and science behind AI Agents](https://newsroom.kireygroup.com/en/news/context-engineering-the-art-and-science-behind-ai-agents?hsLang=en) - [Agentic AI Governance: how to effectively manage AI that can act](https://newsroom.kireygroup.com/en/news/agentic-ai-governance-how-to-effectively-manage-ai-that-can-act?hsLang=en) - [Autonomous SOC: reality, limitations, and prospects for AI-driven security](https://newsroom.kireygroup.com/en/news/autonomous-soc-reality-limitations-and-prospects-for-ai-driven-security?hsLang=en) - [Data Virtualization, the pillar of data-driven companies: what it is and how it works](https://newsroom.kireygroup.com/en/news/data-virtualization-the-pillar-of-data-driven-companies-what-it-is-and-how-it-works?hsLang=en) - [Cloud Economics: the goal is not to spend less, but to spend better](https://newsroom.kireygroup.com/en/news/cloud-economics-the-goal-is-not-to-spend-less-but-to-spend-better?hsLang=en) - [Artificial Intelligence](https://newsroom.kireygroup.com/en/news/tag/artificial-intelligence) / # AI performance: how to measure the success of artificial intelligence initiatives #### Kirey [![](https://newsroom.kireygroup.com/hubfs/it.svg)](https://newsroom.kireygroup.com/news/ai-performance-come-misurare-il-successo-delle-iniziative-di-intelligenza-artificiale?hsLang=it)    [![](https://newsroom.kireygroup.com/hubfs/ES.svg)](https://newsroom.kireygroup.com/?hsLang=es) 26 MIN READ 30 July 2024 - Share on: - - - - [All](https://newsroom.kireygroup.com/news?hsLang=en) - Categories + - [Corporate](https://newsroom.kireygroup.com/en/news/tag/corporate) - [Cybersecurity](https://newsroom.kireygroup.com/en/news/tag/cybersecurity) - [Events/Webinars](https://newsroom.kireygroup.com/en/news/tag/events-webinars) - [Cloud](https://newsroom.kireygroup.com/en/news/tag/cloud) - [Data Value](https://newsroom.kireygroup.com/en/news/tag/data-value) - [Innovation](https://newsroom.kireygroup.com/en/news/tag/innovation) - [Artificial Intelligence](https://newsroom.kireygroup.com/en/news/tag/artificial-intelligence) - [Monitoring](https://newsroom.kireygroup.com/en/news/tag/monitoring) - [DevSecOps](https://newsroom.kireygroup.com/en/news/tag/devsecops) - [Software Development](https://newsroom.kireygroup.com/en/news/tag/software-development) - [Corporate Social Responsibility](https://newsroom.kireygroup.com/en/news/tag/corporate-social-responsibility) - [Data Privacy](https://newsroom.kireygroup.com/en/news/tag/data-privacy) - [Advisory](https://newsroom.kireygroup.com/en/news/tag/advisory) - [Accessibility](https://newsroom.kireygroup.com/en/news/tag/accessibility) - [Automation](https://newsroom.kireygroup.com/en/news/tag/automation) - [Digital Banking](https://newsroom.kireygroup.com/en/news/tag/digital-banking) - [Digital Industry](https://newsroom.kireygroup.com/en/news/tag/digital-industry) - [Use case](https://newsroom.kireygroup.com/en/news/tag/use-case) - [Welfare & Social Services](https://newsroom.kireygroup.com/en/news/tag/welfare-social-services) - [Press & PR](https://newsroom.kireygroup.com/en/news/tag/press) Companies are aware of the virtually **unlimited potential of artificial intelligence**. Credit goes in part to [generative AI](https://newsroom.kireygroup.com/en/news/the-true-opportunity-of-generative-ai-and-how-to-seize-it?hsLang=en), which has extended awareness of AI capabilities beyond industry insiders, but also to the results many companies have already achieved in terms of **efficiency, productivity, automation, and support for strategic and operational decisions.**  **Expectations are very high**. Artificial intelligence can influence competitiveness, but it is crucial that each organization can **measure its actual performance** (AI performance), namely the benefits the company has obtained and is obtaining from its investments. Unfortunately, this is not straightforward.  ## AI performance, asking the right questions  Companies betting on AI must account for a **considerable investment** in skills, infrastructure, technologies and consulting. This creates an immediate need to **demonstrate its value as accurately and quickly as possible**. Only then can investments be justified, new investments secured, stakeholder trust gained, and alignment of ongoing initiatives with business objectives demonstrated.  From this perspective, it might seem that measuring AI performance means **merely calculating the ROI of investments**. However, this is not entirely correct, as **ROI is a central but not exhaustive indicator**. Instead, the company must clearly define the objectives of individual initiatives and adopt **indicators that allow them to monitor the solution's effectiveness and progress made so far**. In other words, the company must first ask the right questions about *how* to evaluate the effectiveness of the solutions being implemented, which naturally depends on their purpose.  - How **has the CX improved** since we introduced artificial intelligence?   - Has **employee turnover** decreased?   - Have **production costs** decreased?   - Has **downtime** decreased?   - How is the solution helping us **achieve our ESG goals**?   - What is the impact on **employee engagement**?   - Is AI actually helping the organization **make better decisions**?  ## Success metrics: from adoption to model accuracy  In the [data-driven era](https://newsroom.kireygroup.com/en/news/data-driven-from-hype-to-reality-how-to-truly-become-an-innovative-company?hsLang=en), measuring success means **defining and tracking specific performance indicators** (KPIs), which once again depend on the initiative's goal and the questions posed. Starting from this premise, it is possible to highlight the most common ones.  ### **Adoption rate**  If AI is integrated into an internal solution, measuring the adoption rate is crucial to **understanding its actual or perceived usefulness**. We talk about *perceived* usefulness because an unsatisfactory adoption rate does not only depend on the solution's ineffectiveness, but also on the company's inability to **manage the complex issue of change** and transform/evolve corporate culture.  ### **Customer experience metrics**  If AI is introduced to enhance customer relations, the primary indicators are those of customer experience (CX). It starts with the traditional NPS (Net Promoter Score), and then delves into more specialized indicators such as CSAT (Customer Satisfaction Score), *churn* *rate*, and many others. Rather than comparing with market benchmarks, **the** **key aspect here is monitoring one's progress over time**.  ### **Employee Experience Assessment**  Maintaining high morale among people is crucial in a productivity-oriented company. Here, artificial intelligence **can act as a** **personal assistant to employees**, help them with many tasks (e.g., searching for complex information), and relieve them from routine operations (e.g., chatbots). In this case, success metrics include *turnover rate*, various engagement KPIs, satisfaction index, and absenteeism rate.  ### **AI model performance**  The term *AI performance* does not only refer to the business effects of the solution but also **the effectiveness**, or rather **the accuracy**, **of the AI technology or model implemented**. Here, various indicators used by data scientists come to the rescue, which are essential for building *trust* in artificial intelligence: **F1 Score, Confusion Matrix, Mean Squared Error (MSE)**, and many others. There are also benchmarks like **Glue** and **SuperGlue** to evaluate specific features such as natural language understanding capabilities.  ## ROI of AI investments, the key indicator  We mentioned that ROI is not the only indicator to monitor, but **it** **remains central** because it measures the profitability of the investment, calculated as the ratio between the invested capital and the realized profit. The ROI of AI investments is an **indicator of the effectiveness of artificial intelligence**, regardless of its purpose.  Calculating AI ROI presents significant challenges, primarily for two reasons:  - The involvement of multiple **intangible elements**;  - The **highly evolutionary nature of AI**. For example, the accuracy of machine learning model results changes over time; this affects the beneficial business effects and, consequently, the ROI of initiatives.  Additionally, it appears that companies make several errors precisely when measuring the return on investment in AI. A few years ago, PwC identified the [top three](https://www.pwc.com/us/en/tech-effect/ai-analytics/artificial-intelligence-roi.html).  - **Not accounting for the uncertainty** due to AI models not having 100% accuracy. Therefore, the **cost of errors** must be considered, which few do;  - **Evaluating ROI in a static way**. As seen, AI performance changes over time, and ROI measurement must take this into account;  - **Evaluating projects as single units** without seeing the *big picture* of the entire portfolio and its effects on the overall business.  ## A path to calculating AI ROI  There are several approaches to calculating return on investment. The problem is not the calculation methodology, for which a **simple *cost-benefit ratio*** or more complex methods such as *payback period calculation* can be used, but rather **identifying all direct, indirect, and somewhat hidden entries** that fall into the macro areas of benefits and costs.  ### **1. Quantification of Tangible and Intangible Benefits**  Quantifying tangible and intangible benefits is a key element in calculating ROI. Tangible benefits are easily quantifiable and measurable in economic terms, such as **operational cost savings and increased direct revenues**.  Intangible benefits are harder to quantify but **equally significant**. Consider, for example, a reduction in product defects, an **enhancement of brand reputation**, or organizational improvement in terms of agility. These are elements that can accelerate (or not) the success of any organization but are **not always translatable into financial terms**.  ### **2. Quantification of investment costs**  Identifying all costs is (obviously) fundamental, but again this is a highly complex factor due to the **number of non-apparent or easily quantifiable items**. For example, the following should be considered:  - Development and implementation costs of the solution;   - Consultancy costs;   - Recurring operational costs, such as maintenance and system updates;   - Change management initiative costs;   - Training costs necessary to fully exploit the solution;   - Integration costs with existing systems;   - Data management costs.  ### **3. Definition of an adequate timeframe**  As mentioned, it is very important to establish the **right moment to measure ROI**, as models and technologies are continually evolving and changing. The timeframe should be selected considering that solutions require a period of adaptation and optimization before producing significant results. This approach **reduces the unrealistic expectations** that unfortunately often accompany the entire topic of artificial intelligence.  ## The importance of measuring AI performance  The promises of AI are fascinating, and there is a risk that companies might adopt the technology without having defined realistic expectations and adequate budgets.  Integrating [artificial intelligence](https://www.kireygroup.com/en/products/artificial-intelligence) into a company **entails significant costs and requires not only skills and technologies but also careful planning**. It is therefore essential to set objectives, define methods to monitor progress, and predict and evaluate actual benefits. Relying on an expert consultant can make a difference.  The topic is constantly evolving, and the above represents only an overview of a much broader and more complex subject that **will become increasingly relevant over time**. However, companies should immediately equip themselves not only on the technical front but also on the **performance evaluation front** to get the most out of it.  - [Previous post](https://newsroom.kireygroup.com/en/news/carlo-alberto-carnevale-maffe-il-nuovo-consigliere-di-amministrazione-di-kirey-group?hsLang=en) - [Read all posts](https://newsroom.kireygroup.com/en/news?hsLang=en) - [Next post](https://newsroom.kireygroup.com/en/news/the-rise-of-hunter-killer-malware?hsLang=en) ## Related posts: #### [Context Engineering, the art and science behind AI...](https://newsroom.kireygroup.com/en/news/context-engineering-the-art-and-science-behind-ai-agents?hsLang=en) With the spread of LLMs, the discipline of prompt engineering emerged, studying the most effective w... [Read more](https://newsroom.kireygroup.com/en/news/context-engineering-the-art-and-science-behind-ai-agents?hsLang=en) 12 MIN READ 30 July 2024 #### [Agentic AI Governance: how to effectively manage A...](https://newsroom.kireygroup.com/en/news/agentic-ai-governance-how-to-effectively-manage-ai-that-can-act?hsLang=en) For years, companies have been asking themselves how to govern AI-generated responses. But what happ... [Read more](https://newsroom.kireygroup.com/en/news/agentic-ai-governance-how-to-effectively-manage-ai-that-can-act?hsLang=en) 10 MIN READ 30 July 2024 #### [AI Act: what happens on 2 August 2026 and what cha...](https://newsroom.kireygroup.com/en/news/ai-act-what-happens-on-2-august-2026-and-what-changes-for-businesses?hsLang=en) 2 August 2026 has always been considered a key milestone in the AI Act implementation timeline. Foll... [Read more](https://newsroom.kireygroup.com/en/news/ai-act-what-happens-on-2-august-2026-and-what-changes-for-businesses?hsLang=en) 8 MIN READ 30 July 2024 #### Explore - [Home](https://www.kireygroup.com/) - [Competencies](https://www.kireygroup.com/en/competencies) - [Industries](https://www.kireygroup.com/en/industries) - [About Us](https://www.kireygroup.com/en/about-us) - [Tech&Partners](https://www.kireygroup.com/en/tech-partners) - [Careers](https://www.kireygroup.com/en/careers) - [Contact](https://www.kireygroup.com/en/contact) - [Corporate Social Responsibility](https://www.kireygroup.com/en/corporate-social-responsibility) - [Advisory](https://www.kireyadvisory.com/en) #### Policy - [Quality](https://www.kireygroup.com/oven/media/PKG-COMP-AL14-Quality-Policy.pdf) - [Security](https://www.kireygroup.com/oven/media/PKG-CIS-LG02-Security-Policy.pdf) - [Gender Equality](https://www.kireygroup.com/oven/media/PKG-COMP-LG03-Politica-per-la-parita-di-Genere-v2.pdf) - [Service Management](https://www.kireygroup.com/oven/media/PKG-COMP-AL45-Politica-per-la-gestione-dei-servizi.pdf) - [Environment](https://www.kireygroup.com/oven/media/PKG-COMP-AL44-Politica-Ambientale-Kirey.pdf) - [Health, Safety & Wellbeing](https://www.kireygroup.com/oven/media/PKG-COMP-LG02-Policy-on-health-safety-and-well-being-in-the-work-environment.pdf) - [ESG](https://www.kireygroup.com/oven/media/Policy-ESG-ENG-v1.pdf) - [Info on Data Processing](https://www.kireygroup.com/en/information-on-data-processing) #### Certifications - [ISO 9001:2015](https://www.kireygroup.com/oven/media/KIREY-S-R-L-9001.pdf) - [ISO/IEC 27001:2022](https://www.kireygroup.com/oven/media/KIREY-S-R-L-27001.pdf) - [PDR 125](https://www.kireygroup.com/oven/media/PDR-125-2022.pdf) - [ISO/IEC 20000-1:2018](https://www.kireygroup.com/oven/media/ISOIEC-20000-12018-ENG.pdf) - [ISO 14001:2015](https://www.kireygroup.com/oven/media/ISO-14001-2015-ENG.pdf) - [ISO 45001:2018](https://www.kireygroup.com/oven/media/CERTIFICATO-ISO-45001-KIREY-ENG.pdf) - [ISO 22301:2019](https://www.kireygroup.com/oven/media/ISO-22301-2019-ENG.pdf) #### Info **Kirey Srl** [info@kireygroup.com](mailto:info@kireygroup.com) Viale Francesco Restelli, 5 20124 Milano [kirey@pec.it](mailto:kirey@pec.it) **PI/CF**: 06729880960 **REA**: MI 1910802 **Tel**: +39 02 78625200 Capitale sociale 1.089.620,00  [![Linkedin](https://newsroom.kireygroup.com/hs-fs/hubfs/Linkedin.png?width=20&name=Linkedin.png)](https://www.linkedin.com/company/kireygroup/)   - [Privacy Policy](https://www.kireygroup.com/oven/media/Kirey-Privacy-ENG-policy-art-13.pdf) - [Cookie Policy](https://www.kireygroup.com/oven/media/Kirey-Cookie-Policy-ENG-1.pdf) - [Terms and Conditions](https://www.kireygroup.com/oven/media/Kirey-Website-Terms-and-Conditions-1.pdf) [![KIREY LOGO](https://newsroom.kireygroup.com/hs-fs/hubfs/KIREY_LOGO_PAYOFF_RGB_POS.png?width=200&height=72&name=KIREY_LOGO_PAYOFF_RGB_POS.png)](https://www.kireygroup.com/) © Copyright 2026 by Kirey