With the spread of LLMs, the discipline of prompt engineering emerged, studying the most effective ways to provide instructions to an AI model and obtain responses that are consistent with the user’s objective. From the very beginning, one principle became clear: the more relevant information and well-defined context the model has access to, the more likely it is to generate a high-quality output.
This is where one of today’s hottest topics comes into play: context engineering.
Key Points
- With the arrival of AI Agents, prompt engineering is no longer enough. The challenge is to design the entire context that will guide the model throughout execution.
- LLMs can process only a limited amount of information before losing precision, making intelligent management of the attention budget essential.
- Context engineering combines context design, dynamic information retrieval, external memory, and multi-agent architectures
Why prompt engineering is not enough
As LLMs have evolved, the quality of instructions (prompt engineering in the strict sense) has become just one part of the equation. Today, the challenge is to effectively manage all the information the model receives while performing a task, i.e. the context in which it operates. This shift has been driven by two phenomena:
- Chats have become much longer and more complex. With each new exchange, the context grows because, quite simply, the model needs to remember previous interactions in order to provide consistent future responses. This makes it increasingly difficult to determine which information is actually relevant to the AI.
- AI agents have arrived. Unlike a traditional chat, an agent does not interact with the LLM just once, but continuously calls it as part of an iterative process. At each step, it consults documents, tools, APIs, and knowledge bases, generates new information, and uses it as context for subsequent steps. The result is an ever-evolving workflow in which the volume of information to be managed grows rapidly.
The challenge stems from the fact that, much like humans, LLMs have limited memory and must operate efficiently. Each model can process only a finite amount of information within its context window, and when that window fills up, the system inevitably loses precision, forgets information that is essential for subsequent decisions, and can no longer assign the appropriate weight to different pieces of information. Since AI agents are intended to support core business processes, losing relevant information or assigning it the wrong weight can result in financial and reputational damage.
The discipline of Context Engineering emerged precisely because providing more context is not enough to obtain better responses: you need to provide the right context at the right time.
What is Context Engineering and how does it optimize the attention budget?
Anthropic defines Context Engineering as the art and science of selecting, organizing, and maintaining the optimal set of information that feeds a language model during each stage of processing. In other words, while prompt engineering focuses on the optimal way to provide instructions, context engineering deals with which information should be made available to the model and when.
The ultimate goal is to make the most efficient possible use of the limited context window of LLMs, ensuring that every token contributes to the quality of the response and avoiding an excess of information that can lead to so-called context rot, i.e. the progressive deterioration of the model’s ability to retrieve the most relevant information.
As mentioned above, context engineering is a dynamic and iterative process because the context evolves based on the tasks performed by the agent and the new information that emerges during the process. The context arises from a combination of three different sources.
User defined instructions
This is the component closest to traditional prompt engineering. It includes system prompts, objectives, constraints, and behavioral rules defined by developers to guide the model.
Information retrieved during execution
While working, an agent continuously retrieves new data by querying external tools and systems. Through protocols such as the Model Context Protocol (MCP), for example, it can access CRM and ERP systems, technical documentation, or web services, loading only the information needed at that particular moment into the context window. The same principle applies to documents, images, or web pages, which are retrieved only when needed, avoiding unnecessary use of space in the context window.
Knowledge produced by the agent
Every search, tool response, intermediate decision, or final output can be useful in subsequent steps. Some systems use agentic memory mechanisms, saving notes and summaries in external memories that can be retrieved when necessary.
How to design an efficient Context Engineering system
In practical terms, context engineering means designing a system capable of providing the model with the right context when needed, regardless of the ad hoc instructions manually provided by the user to the model through prompts. Achieving this requires intervention across several aspects of agent architecture.
Design an essential context
The first principle is to avoid wasting the attention budget. System prompts, instructions, examples, and constraints should be detailed enough to guide behavior, but without going too far. Otherwise, they risk unnecessarily occupying the entire context window. The solution lies in finding the right balance between hard-coding every single rule and relying on generic statements with little real value.
Design token-efficient tools
The tools used by AI agents must also be designed with the need to interact with a limited context window in mind. The goal is to ensure that each integration returns only the information that is necessary. A well-designed tool does not return hundreds of records when three are enough, nor entire documents when an excerpt will do.
Retrieve context progressively
Preloading all the information available to the agent is not a good practice. A more effective approach is what Anthropic calls progressive disclosure: the agent can, for example, view only the metadata of a set of documents, decide which ones to investigate further, and retrieve the full content only when necessary. This behavior is defined by developers through the design of workflows, APIs, and the rules governing information integration.
Manage long-running tasks
The most complex tasks can far exceed the context window capacity of an LLM. For this reason, more advanced architectures adopt strategies such as summarization, which automatically summarizes the work performed while preserving essential decisions and information with a significant reduction in space. Another possibility is to use multi-agent architectures, which distribute the work among specialized agents, each with its own context window, while delegating to a coordinating agent the task of collecting and synthesizing the results.
Take on the challenges of Agentic AI together with us
At Kirey, we help companies make the most of the potential of artificial intelligence. In recent months, AI Agents have become the main driver of this evolution due to their ability to reason, make decisions, and act autonomously on business processes. This certainly has the potential to transform the way companies operate.
The opportunities are enormous, but the technical and design challenges to be addressed are equally concrete, especially in a field that is evolving at an unprecedented pace.
It is through challenges such as the context window that we support companies in their transformation, helping them improve efficiency and productivity, accelerate innovation, and build the capabilities they need today to address tomorrow’s demands.
Contact us to discover how to bring Agentic AI into your organization’s processes.
