Artificial intelligence has an innovative side, but it also comes with a bill to pay. And in recent months, that bill has begun to concern CIOs around the world, who are already grappling with the longstanding challenge of demonstrating the ROI of their AI investments.
Key Points
- The unit cost of tokens is falling, but the complexity and consumption of agentic workflows are increasing faster. Many companies are significantly exceeding their planned budgets.
- Reasoning, repeated calls, extended contexts, external tools, and autonomy all add up. There is no single cause, but rather a chain of cost multipliers that are often difficult to predict.
- Agents must be designed not only to achieve their objectives, but also to operate efficiently. This is possible, provided the necessary skills and experience are available.
The Inference Paradox: companies are spending more than $1 million a year
In August 2026, Gartner coined the term the inference paradox, describing it as follows: “By 2028, inference costs per agentic workflow will increase more than fivefold.” The paradox lies in the fact that the unit cost of calls, measured in tokens, continues to fall, while companies’ overall spending keeps rising.
But how much are businesses spending? According to IDC, companies with visibility into their costs report average monthly spending of $117,558 on inference and orchestration services, or more than $1.4 million a year. And this is only the beginning.
Perhaps the most significant finding, however, concerns the inability to forecast spending: according to IDC, over the past twelve months, 67% of companies have exceeded their budget for AI agents by more than 10%; 24% have overshot it by an even greater margin.
And when budgets are blown, companies respond in the most straightforward way possible: they postpone other IT projects, and 43.1% reduce headcount to offset the additional spending.
It is clearly time to look for a third way, starting from the premise that AI is not expensive because each token costs too much, but because an apparently simple task sets an entire chain of consumption in motion.
An agentic workflow can consume 1,000 times more tokens than a chat
For a CIO, the difference between a chat and a workflow is becoming increasingly clear. In a standard chat, the model receives a question, draws on its knowledge, and produces an answer. An agentic workflow must reason, plan, use tools, analyze results, check its own work, and potentially retrace its steps.
Anyone who has experimented with the reasoning capabilities of AI models will have noticed that, as complexity increases, processing times and token consumption increase too, driving up costs. But to understand where that money actually goes, we need to dig deeper.
The model consumes resources multiple times
The first multiplier is internal to a single call. When faced with a complex problem, the model does not immediately produce an answer. Instead, it uses reasoning capabilities to analyze information, evaluate alternatives, and construct the result. This processing requires additional tokens, even though, from the user's perspective, everything appears to happen in response to a single prompt. The easiest way to notice this is through processing times, which are entirely incompatible with those of a standard chat.
The second multiplier, by contrast, depends on the workflow's structure. Consider the processing of an incoming invoice. The model extracts the supplier, amount, and purchase order number from the document. Based on this data, the workflow queries the ERP system, retrieves the supplier's master data, and checks that the goods and amounts match the order. If a discrepancy emerges, the system sends the model information it did not previously have and asks it to interpret the discrepancy, decide which additional checks to perform, or prepare a request for clarification. The resulting answer may trigger further checks and, consequently, additional calls. It is this back-and-forth between reasoning, access to enterprise systems, and successive results that turns a single user request into a sequence of inference operations.
As McKinsey observes, “the most expensive part is not generating the first response, but the subsequent process of checking, correcting, and checking again.” In other words, producing an answer is relatively inexpensive; making it reliable is much less so.
Autonomy is not always a good thing
CIOs need predictable costs. AI models and agentic workflows, if not properly governed, move in the opposite direction. An agent does not necessarily follow the same path as another agent performing a similar task: it may develop different plans, choose different tools, repeat a search, revise a decision, or get trapped in a loop of repeated attempts.
The result is that two executions of the same task can have radically different costs. A 2026 study conducted on eight frontier models found variations of up to 30 times in token consumption for the same task. The same models also proved poor at predicting their own consumption in advance and tended to underestimate it.
There is a second issue, too: using more tokens does not guarantee greater accuracy. Performance often reaches its optimum at an intermediate level of consumption, then plateaus despite further spending. In the agentic world, therefore, adding steps, checks, and reasoning does not necessarily lead to better results. Complexity can turn into consumption without value.
Inference is not the only cost
An agentic workflow does not rely solely on the AI model. It may perform searches, query databases, execute code, generate images, or access SaaS platforms. Each call therefore brings additional costs: APIs, computing capacity, licenses, storage, and data transfer. The cost of a workflow is the sum of all these individual charges, not just the tokens.
Speed and availability come at a cost, even when they are not used
AI agents are entering critical business processes, where delivering a high-quality result is not enough: it must arrive quickly, and the service must remain available at all times. Ensuring consistent response times, even during peak periods, requires reserved computing capacity, redundant infrastructure, and sufficient headroom to absorb sudden increases in demand. The company continues to pay for these resources even when traffic declines, and they remain partially unused.
Architecture determines the cost of AI agents
There is no single validated technique for solving the inference paradox. The fundamental rule is to design agents so that they use AI only when necessary, to the extent required and within clearly defined boundaries. Efficiency cannot be added at the end: it must be built into the workflow architecture from the outset. Otherwise, an unpleasant surprise is almost inevitable.
Don't use AI when a database is enough
Calculations, filters, comparisons, and deterministic rules do not require a language model. If an operation can be performed with an SQL query or a few lines of code, delegating it to AI makes it more expensive, slower, less predictable, and less reliable. The model should step in when interpretation, reasoning, or ambiguity management is required.
Assign the right model to each task
Few tasks require the most powerful and expensive model. Extraction, classification, and summarization can be handled by lightweight models, while complex reasoning and high-risk decisions may justify the use of more advanced ones. Clear guidelines are essential, along with a routing system that selects the model based on the complexity, value, and risk of each individual task.
Don’t pay for the same work more than once
An efficient agent must proceed incrementally, retaining intermediate results instead of reconstructing them at every step. It may sound obvious, but without clearly defined rules, the model repeats the entire cycle. Stable information can be stored or cached; temporary information can be summarized or discarded when no longer needed. This way, the model receives only the context required for the current decision.
Make the cost of every task visible
With thousands of agents and numerous reasoning steps for each one, knowing total spending is not enough. Tokens, calls, tools, and retries must be attributed to the smallest measurable unit of work. If it is impossible to identify which task is exceeding its budget, the problem cannot be corrected. This is where AI FinOps comes into play.
Set boundaries on autonomy
An agent should not be allowed to reason, use tools, or revise its results indefinitely. Every workflow must include limits on time, tokens, iterations, and attempts. Once a threshold is exceeded, the agent must stop, adopt an alternative strategy, or hand the case over to a person.
Find the right balance between cost and quality
More reasoning, more checks, and a more powerful model do not automatically produce better results. It is necessary to experiment with different configurations and measure the cost of each completed task for each one. The goal is to identify the point beyond which increased consumption no longer delivers a meaningful quality improvement.
Innovation means turning AI into value, not cost
At Kirey, we operate at the forefront of technological innovation, which increasingly relies on AI agents. We support companies in automating and evolving their processes, leveraging the most advanced technologies and turning our expertise into a tangible competitive advantage.
However, an advantage cannot truly be called one if the return on investment remains uncertain. That is why, alongside the many challenges posed by agentic AI, we pay particular attention to inference costs. Our goal is to prevent falling token prices from paradoxically translating into higher overall spending.
Contact us to embark on an innovation journey together, combining the potential of AI with control, economic sustainability, and measurable results.
