Agent Harness · Agentic AI · MCP
Agent harness engineering: a production guide
Agent = model + harness. A production guide to agent harness engineering: tool loops, context, state, sandboxes, guardrails, MCP and evals, from real agents.
· 6 min read
Two teams can use the same frontier model and get completely different agents. The difference is the harness — the tools, context, state, guardrails and checks wrapped around the model. This guide breaks a production harness into seven layers and shows how I implement each one in enterprise agents.
What is an agent harness?
An agent harness is the runtime around a large language model that turns its reasoning into controlled action. The model decides what to do next; the harness executes tools, manages what the model sees, persists state, enforces permissions, checks results and records everything. The industry shorthand is simple: agent = model + harness.
Claude Code, Cursor and the agent SDKs from Anthropic, OpenAI and Google are all harnesses. When you build an enterprise agent with LangGraph, you are building one too — whether or not you design it deliberately.
Why does the harness matter more than the model?
Because most agent failures in production aren’t intelligence failures. They are tools called with malformed arguments, plans lost after a long conversation, actions taken on stale state, or results nobody verified. Swapping in a newer model rarely fixes those; changing the harness does. Published harness-only changes — restructured tool formats, injected context, self-verification hooks — have produced large benchmark gains with the model held constant.
There is a practical consequence for teams: models change every few months, harnesses compound. Every bug you fix in the harness — a sharper tool description, a new guardrail, an extra check — keeps paying off when you upgrade the model underneath.
The seven layers of a production agent harness
I design and review harnesses layer by layer. Skipping any one of them is where incidents come from.
| Layer | What it controls | How I implement it |
|---|---|---|
| Execution environment | Where the agent's actions actually run | Isolated sandboxes or containers for code and CLI execution, allow-listed commands, network egress limits, ephemeral workspaces per task. |
| Tool interface | How the model discovers and calls capabilities | Typed tools with strict JSON schemas, timeouts and retries; clear descriptions written for the model; MCP servers for integrations reused across agents and clients. |
| Context management | What the model sees on each step | Context engineering: load only what the current step needs, retrieve on demand (RAG), summarise or compact long histories, keep instructions in versioned files. |
| State & orchestration | How work survives across steps, failures and handoffs | LangGraph state machines with checkpointing, explicit phases, supervisor-plus-specialist routing, sub-agents with narrow scopes, durable progress records. |
| Guardrails & permissions | What the agent is allowed to do | Least-privilege credentials kept out of the model's context, read-only by default, human-in-the-loop approval before irreversible actions, prompt-injection-aware handling of tool output. |
| Verification | How the agent knows it got it right | Validators, schema checks, dry runs and test suites wired in as hooks; failures are fed back to the model with the exact error so it can self-correct. |
| Observability & evals | How you know the system is improving | Full traces of prompts, tool calls, cost and latency in Langfuse or LangSmith; golden eval datasets and regression suites in CI on every prompt, tool or model change. |
Case study: the harness behind an infrastructure-provisioning agent
My OCI Terraform Agent lets users describe cloud infrastructure in plain English and have it provisioned. An agent that can create real cloud resources is only safe because of its harness:
- Requirement gathering is its own graph phase: the agent asks clarifying questions until the infrastructure request is complete, and phase-aware context tracking makes sure a follow-up message lands at the right node.
- Intent routing covers 9+ intent types — creating infrastructure, querying resources, pushing Docker images to OCIR, editing Terraform — each with its own tools rather than one do-everything prompt.
- The agent generates a full Terraform plan with cost estimates and stops. Nothing is applied until a human approves the plan; only then does the harness run terraform init, plan and apply, streaming output live over WebSockets.
- Cloud credentials are Fernet-encrypted at rest and injected by the harness at execution time. The model never sees them, so a prompt-injection or a leaked transcript can't exfiltrate them.
- FastAPI and WebSockets carry the loop; PostgreSQL holds conversation and workflow state so sessions resume after a restart.
The same pattern — narrow phases, typed tools, approval before side effects, verification and tracing — carries over to the multi-agent tutoring platform and the Text-to-SQL analytics agent described in my project case studies, and to voice agents: see the voice AI lead generation platform architecture.
Which framework should you build your agent harness on?
Pick the framework that gives you the loop you need, then invest in the layers it doesn’t give you. My rule of thumb:
- LangGraph: You need explicit control flow: phases, branches, checkpoints, interrupts for approval and multi-agent routing. My default for complex enterprise workflows.
- Claude Agent SDK: You want a ready-made agent loop with file system, tool use, sub-agents and context compaction, and Claude is your primary model.
- OpenAI Agents SDK / Google ADK: Your organisation is standardised on that platform and wants its native tracing, handoffs and deployment story.
- Pydantic AI / LlamaIndex: Type-safe, Python-first agents (Pydantic AI) or retrieval-heavy agents over large document collections (LlamaIndex).
Whichever you choose, expose integrations as MCP servers where they will be reused, and keep the model behind a gateway — Claude, Gemini or Bedrock-hosted models — so the harness outlives any single model. For orchestration patterns in depth, see agentic AI systems architecture with LangGraph; for the service layer, see integrating LLMs into enterprise microservices.
Production agent harness checklist
- Every tool has a typed schema, a timeout, a clear model-facing description and a test.
- The context for each step is assembled deliberately — no full-history dumps.
- State is checkpointed; a crash or restart resumes the workflow instead of starting over.
- Irreversible or expensive actions require human approval, and the approval is logged.
- Secrets never enter the model's context; tool output is treated as untrusted input.
- There is an automated way to verify the result of each critical step.
- Every run is traced with cost and latency; evals run in CI before any prompt, tool or model change ships.
- The harness is model-agnostic behind one gateway, so switching models is a config change.
Frequently asked questions
What is an agent harness?
An agent harness (also called agent scaffolding) is the software around a large language model that lets it act as an agent: the tool-execution loop, context management, memory and state, sandboxed execution, permissions, verification and observability. A raw model only produces text; the harness turns that text into controlled actions.
What is the difference between harness engineering and prompt engineering?
Prompt engineering optimises the instructions given to the model. Context engineering optimises everything placed in the model's context window. Harness engineering is the widest scope: it designs the whole runtime around the model, including tools, state, sandboxes, guardrails, verification hooks and evals.
Do I need a framework to build an agent harness?
No, but you shouldn't rebuild the loop from scratch without a reason. LangGraph, the Claude Agent SDK, the OpenAI Agents SDK and Google ADK all provide a loop and tool calling. The differentiating work is the layers you add on top: permissions, verification, context policy and evals specific to your domain.
How does MCP fit into an agent harness?
MCP (Model Context Protocol) is the tool-interface layer. Exposing a system as an MCP server lets any MCP-capable agent or client use it with the same schemas and auth, so you build each integration once. The harness still decides which servers are allowed and which actions need approval.
References and further reading
- LangChain — The anatomy of an agent harness
- Anthropic Engineering — Effective harnesses for long-running agents
- O'Reilly Radar — Agent harness engineering
- Model Context Protocol specification
Related: Retrieval for agents: vector databases.