Context Engineering: Concepts, Problems, and Lessons

The Beginning of the Context Engineering Discussion

On June 19, 2025, Shopify CEO Tobi Lütke mentioned the term “context engineering” on Twitter: “I really like the term ‘context engineering’ over prompt engineering. It describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM.”

As LLM context windows have become longer—from 2K tokens in ChatGPT-3.5 to 1M in Gemini, a 500-fold increase—models have been asked to perform increasingly difficult tasks. This has made it important to fill the context window with the reference material needed to produce the required output.

Andrej Karpathy later quoted the post and contrasted context engineering with prompt engineering. His point was that industrial-strength LLM applications require the delicate art and science of filling the context window. Too little information, or incorrect information, prevents the LLM from doing its job properly. But including too much unnecessary material increases cost and can reduce performance.

Why Context Engineering Rather Than Prompt Engineering?

Many people say that it is important to make AI work well by asking good questions. For ordinary chatbot use in a question-and-answer format, this is naturally true. Prompt engineering is worth studying and remains an important skill: the tokens entered by a human matter.

However, reasoning models that use long chains of reasoning to answer difficult questions, and agentic models that call multiple tools to execute complex plans step by step, are becoming more common. In these cases, the tokens generated by the AI during intermediate stages become more important than the user’s initial input.

Over a long task, the AI accumulates far more tokens in context than a human initially provides, and those tokens have a larger effect on the result. An LLM agent is LLM-based software that autonomously plans and uses tools to complete a difficult task through many steps. To decide what to do next and produce a result, it generates and refers to substantial amounts of information: an overall plan, context retrieved through RAG, instructions for tools such as MCP documentation, outputs from earlier stages, and reasoning tokens.

If this accumulated context includes incorrect or unnecessary content, performance declines.

Four Types of Context Failure

I came across Drew Brunig’s writing on this subject and found the framework persuasive. The following are four modes of context failure:

• Context poisoning: Incorrect content is generated, especially early in a task, and is repeatedly referenced in later work.
• Context distraction: When context becomes too long, the model spends too much effort attending to it, reducing its ability to use its own knowledge or generate new ideas.
• Context confusion: Unnecessary material reduces accuracy or efficiency.
• Context clash: Conflicting information creates confusion and leads to inconsistent answers.

Ways to Address Context Failure

• Perform RAG well. A larger context window does not mean everything should be thrown into it. Providing only the context needed for the task is better for both performance and efficiency.
• Select tools carefully. Unless a model has been fine-tuned, the list of available tools and their instructions—such as MCP documentation—must be added to its context. It is well known that too many tools can create problems. The idea is to retrieve information only about the tools needed for the specific task: in effect, RAG for MCP or tool descriptions.
• Isolate context. Divide a large task into subtasks that can be completed independently in separate contexts, then parallelize them. Claude’s Deep Research is said to be highly efficient compared with competing services because it uses a multi-agent research system in which an orchestrator distributes work among multiple subagents.
• Prune context. Rather than continually accumulating context throughout a multi-step task, remove knowledge and logs that are no longer needed for the next step. This can address context poisoning and context confusion.
• Summarize context. As work accumulates, the context can become too long. It may be possible to preserve the information while compressing it into a denser, more organized summary. This addresses context distraction.
• Offload context. If information is not needed at every stage, store it elsewhere and retrieve it only when needed. Anthropic’s think tool is similar in concept, though this refers not to extended thinking that adds a reasoning process. For example, tool execution results can be stored separately and inserted into the context window only when needed; important guidelines can be retrieved for an intermediate review; or the output of a particular step can be brought back for close comparison.

What General Users Can Take From This

Even non-developers increasingly work through long conversations with LLM chatbots. At recent talks, I have repeatedly been asked: “In a long conversation, the chatbot becomes slower and starts saying strange things. What should I do?” My suggestions are as follows.

• Use the edit feature actively. As a conversation continues, unnecessary content may be added, creating context confusion, or an unwanted and incorrect answer may enter the conversation, creating context poisoning. Instead of adding a new instruction, edit the earlier content to remove incorrect material or create a new branch.
• Create a summary. If the conversation becomes too long and causes context distraction, ask the chatbot to prepare a document summarizing the conversation so far. Then start a new conversation, attach the summary, and resume the work with a more compact context.

Conclusion

Beyond prompt engineering—the idea that one simply needs to ask good questions—it may be useful to understand context engineering: managing the tokens, or context, that accumulate while work proceeds through a conversation with a chatbot. This can make it easier to work with LLM chatbots.

I seem to have written a paper while writing this.