Harness Engineering: The Complete Guide to Building AI Agents That Don't Fall Apart

Most people are trying to improve AI agents at the wrong layer.

When an agent fails, they rewrite the prompt.

When it fails again, they add more instructions.

Before we start:

Follow my Substack for fresh AI alpha, agent workflows, and step-by-step guides before they hit X: https://substack.com/@lunarresearcher

Then they switch models, add more tools, increase the context window, and hope the next run behaves differently.

But many agent failures are not reasoning failures.

They are environment failures.

The agent did not know which files mattered.

It used the right tool in the wrong place.

It lost the decisions made in the previous session.

It claimed success without running the checks.

It repeated an action after a partial failure.

It had permission to do something that should have required approval.

The model was not necessarily the problem. The system around the model was incomplete.

That system is the harness.

And designing it is becoming its own engineering discipline.

Harness Engineering is the practice of building the environment that turns model intelligence into reliable work.

A prompt changes one attempt.

A harness changes every attempt.

This guide explains how to build one.

1. The Model Is Not the Agent

A model can reason, generate, compare, and choose.

But an agent must also interact with a real environment.

It needs to:

• understand the task
• find the relevant context
• select and use tools
• preserve state
• respect permissions
• inspect the result
• recover from failure
• prove that the work is complete

The model is the reasoning engine inside that system.

The harness is everything that makes the reasoning operational.

1user request
2     |
3     v
4+-----------------------------+
5|           HARNESS           |
6| contract | context | policy |
7| tools    | state   | checks |
8| traces   | recovery          |
9+-----------------------------+
10     |
11     v
12    model
13     |
14     v
15real environment

A powerful model inside a weak harness is still a weak agent.

It may produce impressive individual responses, but it will behave inconsistently across long tasks, changing environments, and partial failures.

The goal of harness engineering is not to remove uncertainty from the model.

It is to contain that uncertainty inside a system that can observe, verify, and recover.

2. Start With a Task Contract

Most agent tasks begin as vague intent:

Improve the onboarding flow.

That sentence may be enough for a conversation.

It is not enough for autonomous execution.

Before the agent acts, the harness should convert the request into a task contract.

A useful contract answers five questions:

1. What outcome must exist?
2. What is inside the scope?
3. What must not change?
4. What evidence proves completion?
5. Which actions require human approval?

1objective: reduce onboarding drop-off
2
3scope:
4  - signup flow
5  - onboarding analytics
6
7constraints:
8  - do not change authentication
9  - preserve existing mobile behavior
10
11acceptance:
12  - tests pass
13  - analytics event is emitted
14  - screenshots cover desktop and mobile
15
16approval_required:
17  - production deployment
18  - database migration

This changes the agent's question from:

What should I do next?

to:

What action moves the environment toward the contracted outcome?

Without a contract, the agent optimizes for plausible activity.

With a contract, it can optimize for verified completion.

3. Give the Agent a Map, Not a Manual

Dumping the entire repository, documentation set, and conversation history into context is not good context engineering.

It is context flooding.

The harness should provide a small map first, then let the agent retrieve details when they become relevant.

1PROJECT MAP
2
3product rules     -> docs/product/
4architecture      -> docs/architecture.md
5frontend          -> apps/web/
6backend           -> services/api/
7tests             -> tests/
8commands          -> docs/commands.md
9release rules     -> docs/release.md

This is progressive disclosure:

1task
2  -> project map
3      -> relevant subsystem
4          -> exact files
5              -> local instructions

The context should expand because the task requires it, not because the information exists.

A good context compiler decides:

• what is always needed
• what can be retrieved later
• what has become stale
• what can be summarized
• what must remain verbatim

The objective is not maximum context.

It is maximum signal per token.

4. Build a Tool Gateway, Not a Tool Pile

Giving an agent twenty tools does not make it capable.

It gives the agent twenty ways to make a mistake.

Every tool should have a clear contract:

1TOOL: edit_file
2
3inputs:
4  path
5  patch
6
7preconditions:
8  path exists
9  path is inside allowed workspace
10
11success evidence:
12  patch applied
13  resulting diff returned
14
15failure behavior:
16  no partial overwrite
17  structured error returned
18
19risk class:
20  reversible

The harness should control how tools are exposed and used.

It can:

• hide irrelevant tools
• validate arguments
• restrict paths and domains
• attach timeouts
• make retries idempotent
• normalize outputs
• require confirmation for risky actions
• return evidence, not just "success"

This creates an important separation:

1model decides intent
2gateway validates action
3tool changes environment
4sensor observes result

The model can propose an action.

The tool gateway decides whether that action is valid enough to execute.

5. Separate the Brain, the Hands, and the History

Many fragile agents mix everything into one growing transcript.

Reasoning, tool calls, files, decisions, errors, and old observations all compete for the same context window.

A stronger system separates three responsibilities:

1BRAIN
2plans, reasons, chooses
3
4HANDS
5execute tools inside a controlled environment
6
7HISTORY
8stores durable facts, decisions, and run state

The model does not need every raw event in active context.

It needs the right current state.

The sandbox does not need to understand the entire objective.

It needs to safely execute a bounded action.

The session log does not need to reason.

It needs to preserve what happened after the current context disappears.

This separation makes long-running agents easier to resume, inspect, and repair.

It also lets you replace one part without rebuilding the entire system.

6. Memory Must Become Durable State

Conversation history is not reliable memory.

It is an event stream.

Useful memory should be converted into explicit state.

At minimum, preserve four categories:

1FACTS
2stable information discovered about the environment
3
4DECISIONS
5choices made and the reason behind them
6
7PROGRESS
8completed, active, blocked, and remaining work
9
10LESSONS
11failures that should change future behavior

For example:

1facts:
2  - checkout validation lives in services/orders
3
4decisions:
5  - reuse the existing validation pipeline
6  - reason: avoids a second source of truth
7
8progress:
9  completed:
10    - added server-side rule
11  remaining:
12    - update integration test
13
14lessons:
15  - local test command requires TEST_DB_URL

This is far more useful than replaying fifty pages of transcript and hoping the model notices the important line.

Store raw history for auditability.

Compile durable state for execution.

7. Completion Requires Evidence

An agent saying "done" is not evidence that the task is done.

It is only another model output.

Completion must be decided by observable changes in the environment.

1claim                         evidence
2--------------------------------------------------
3"the bug is fixed"            failing test now passes
4"the page works"              browser flow completed
5"the migration is safe"       dry run and rollback pass
6"the report is correct"       values match source data
7"the task is complete"        every acceptance check passes

The harness should run the cheapest deterministic checks first.

1syntax
2  -> types
3      -> focused tests
4          -> integration tests
5              -> visual or semantic review
6                  -> human approval

Do not use another model where a compiler, schema, checksum, query, or test can answer the question.

Use models for ambiguity.

Use code for plumbing.

A model can propose that the task is complete.

Only the environment can prove it.

8. Verification Should Attack the Result

Workers and evaluators should not share the same objective.

The worker tries to create the strongest solution.

The evaluator tries to find the reason it should be rejected.

1worker
2  -> produces candidate
3
4verifier
5  -> checks contract
6  -> searches for missing cases
7  -> tests unsupported claims
8  -> attempts to break result
9
10survives
11  -> accept
12
13fails
14  -> return targeted evidence

This asymmetry matters.

If you ask the same agent, in the same context, to "double-check its work," it often preserves the assumptions that created the mistake.

A useful verification stage should have:

• an explicit rejection rubric
• access to the produced artifact
• access to the acceptance contract
• independent tools or fresh context when needed
• permission to reject without repairing

Verification is not a second opinion.

It is an attempted disproof.

9. The Model Proposes, the Policy Authorizes

Some rules should never depend on whether the model remembers them.

1never publish without approval
2never expose a secret
3never write outside the workspace
4never exceed the spend cap
5never mark tests passed unless they ran

These are not prompt suggestions.

They are policy.

The safest design keeps policy outside the reasoning loop.

1LOW RISK
2read files, search, inspect
3-> automatic
4
5REVERSIBLE CHANGE
6edit workspace, run tests
7-> automatic with trace
8
9EXTERNAL EFFECT
10send message, deploy, purchase
11-> explicit approval
12
13IRREVERSIBLE OR SENSITIVE
14delete data, rotate credentials, publish globally
15-> hard gate or prohibited

The stronger the consequence, the harder the gate.

Autonomy is not the absence of control.

It is the ability to operate freely inside a clearly enforced boundary.

10. Recovery Should Target the Failure Class

The most common recovery strategy is:

Something failed. Try again.

That is not recovery.

It is repetition.

The harness should classify the failure before selecting the next action.

1tool timeout
2-> retry with backoff
3
4invalid arguments
5-> repair the tool call
6
7missing context
8-> retrieve specific source
9
10failed test
11-> inspect failing behavior
12
13permission denied
14-> request approval or choose safe path
15
16contradictory requirements
17-> escalate to human
18
19repeated unchanged failure
20-> stop the loop

A retry should change at least one relevant condition.

Otherwise the system is paying to reproduce the same failure.

A bounded agent loop looks like this:

1observe
2  -> decide
3      -> act
4          -> measure
5              -> accept
6              -> repair
7              -> escalate
8              -> stop

Every loop needs a budget:

• maximum attempts
• maximum time
• maximum spend
• maximum destructive scope
• escalation condition

Reliable agents know how to continue.

They also know when continuing is no longer rational.

11. Instructions Should Become Infrastructure

Agent instructions are useful when they explain local reality.

But instructions alone are weak enforcement.

If a rule matters repeatedly, move it down the stack.

1"use the formatter"
2-> run formatter automatically
3
4"do not import across layers"
5-> add architecture test
6
7"include a migration rollback"
8-> require rollback file in CI
9
10"do not modify generated files"
11-> block writes to generated paths
12
13"cite every external claim"
14-> validate citation coverage

This creates an instruction ladder:

1explanation
2  -> checklist
3      -> template
4          -> automated check
5              -> enforced policy

Move important knowledge as far down that ladder as practical.

The prompt should explain judgment.

The harness should enforce invariants.

12. Observe the Run, Not Just the Final Answer

A clean final artifact can hide a terrible process.

The agent may have:

• accessed the wrong data
• ignored a failed command
• retried an external action twice
• consumed ten times the expected budget
• reached the right answer for the wrong reason

You need traces that make the run reconstructable.

109:14  contract created
209:15  context source loaded: architecture.md
309:17  file edited: checkout.ts
409:18  focused test failed: duplicate coupon
509:21  implementation repaired
609:22  focused test passed
709:24  integration test passed
809:25  external deployment blocked: approval required

A useful trace records:

• state transitions
• context sources
• tool inputs and outputs
• environment changes
• verification results
• retry reasons
• approval decisions
• cost and latency

The goal is not surveillance.

The goal is local repair.

When a run fails at step 18, you should be able to restart from a trustworthy checkpoint instead of replaying the entire task.

13. Every Run Needs a Change Receipt

Long agent transcripts are difficult to review.

At the end of a run, the harness should compile a small change receipt.

1OBJECTIVE
2Fix duplicate coupon application during checkout.
3
4CHANGED
5- checkout validation logic
6- focused regression test
7
8VERIFIED
9- lint passed
10- unit tests passed
11- checkout integration test passed
12
13NOT VERIFIED
14- production payment provider
15
16DECISIONS
17- preserved existing coupon priority order
18
19RISKS
20- legacy mobile client was not available locally
21
22APPROVAL NEEDED
23- deploy to staging

The receipt is not a summary of what the model said.

It is a summary of what the system can prove.

This gives humans a compact review surface and gives the next agent session a trustworthy starting point.

The best handoff is not "here is the conversation."

It is "here is the state, the evidence, and the unresolved risk."

14. Every Failure Should Upgrade the Harness

The weakest teams fix the failed output.

The strongest teams also fix the system that allowed it.

After a failure, ask:

1Was the task contract ambiguous?
2Was important context invisible?
3Was the wrong tool exposed?
4Was a precondition missing?
5Was the result unverifiable?
6Was policy left inside the prompt?
7Was recovery too broad?
8Was the trace insufficient?

Then convert the lesson into a reusable improvement.

1failure
2  -> diagnosis
3      -> new sensor, rule, map, test, or tool contract
4          -> future runs improve automatically

This is the harness flywheel.

The system becomes more reliable because failures leave infrastructure behind.

A corrected answer helps one run.

A corrected harness helps every future run.

15. Harnesses Decay Too

More harness is not always better.

Models improve. Tools improve. Tasks change. Old safeguards can become unnecessary friction.

A workaround created for yesterday's model may prevent today's model from using a better strategy.

This creates harness decay:

1old model limitation
2  -> harness workaround
3      -> model improves
4          -> workaround remains
5              -> system becomes slower or less capable

Treat harness components like production code.

Measure whether they still provide lift.

For every router, evaluator, memory layer, and retry rule, ask:

• Which failure does this prevent?
• How often does that failure still occur?
• What latency and complexity does this add?
• Can the same result now be achieved more simply?
• What happens if we remove it?

The best harness is not the largest one.

It is the smallest system that reliably closes the gap between intent and evidence.

Build to delete.

16. The Minimum Viable Harness

You do not need an orchestration platform to begin.

Build the harness in layers.

Level 1: A bounded task

• objective
• scope
• constraints
• acceptance checks

Level 2: A legible environment

• project map
• commands
• local instructions
• known dependencies

Level 3: Controlled actions

• typed tools
• argument validation
• path and permission boundaries
• structured results

Level 4: Durable execution

• explicit run state
• checkpoints
• decisions
• lessons

Level 5: Evidence

• deterministic checks
• adversarial verification
• change receipt

Level 6: Recovery and learning

• failure classification
• bounded retries
• escalation
• harness updates from recurring failures

Build the smallest layer that eliminates the failure you actually have.

Do not begin with a multi-agent architecture because a single prompt occasionally needs clarification.

Complexity should be earned by observed failure.

17. A Reusable Harness Specification

Before giving an agent meaningful autonomy, define this:

1AGENT HARNESS SPEC
2
31. CONTRACT
4   objective:
5   scope:
6   constraints:
7   acceptance evidence:
8
92. CONTEXT
10   always-loaded map:
11   retrieval sources:
12   local instructions:
13   freshness rules:
14
153. TOOLS
16   allowed tools:
17   preconditions:
18   side effects:
19   success evidence:
20   timeout and retry policy:
21
224. STATE
23   facts:
24   decisions:
25   progress:
26   lessons:
27   checkpoint format:
28
295. POLICY
30   automatic actions:
31   approval-required actions:
32   prohibited actions:
33   budget limits:
34
356. VERIFICATION
36   deterministic checks:
37   adversarial checks:
38   acceptance rule:
39
407. RECOVERY
41   failure classes:
42   retry limits:
43   escalation conditions:
44   safe rollback:
45
468. OBSERVABILITY
47   trace events:
48   metrics:
49   final change receipt:

If these fields are undefined, the agent is not autonomous.

It is improvising.

18. Measure the System at the Right Level

Token count is not the final metric.

Neither is the number of tasks attempted.

The useful unit is accepted work.

A practical metric is:

1accepted outputs
2------------------------------
3human review minutes + run cost

Also track:

• first-pass acceptance rate
• recovery rate after tool failure
• repeated failure rate
• human interventions per task
• unsupported completion claims
• time from request to verified outcome
• harness overhead by component

This prevents a common illusion:

An agent can look highly productive while creating expensive review work.

The objective is not more agent activity.

It is more trusted outcomes per unit of human attention.

19. When You Do Not Need a Heavy Harness

Not every model call needs an operating system.

Use a simple prompt when:

• the task is short
• the output is easy to inspect
• failure is cheap
• no external side effect occurs
• the user remains in the loop

Add a harness when:

• work spans multiple tools or sessions
• the environment can change
• actions have real consequences
• completion is difficult to judge manually
• the same failure appears repeatedly
• human review becomes the bottleneck

The purpose of a harness is not to make a demo look sophisticated.

It is to make real work dependable.

The Real Shift

The first generation of AI products was built around prompts.

The next generation is being built around environments.

The question is no longer only:

How do we make the model answer better?

It is:

How do we build a system where good actions are easy, dangerous actions are controlled, failures are visible, and completion is provable?

That is the shift from prompt engineering to harness engineering.

The model supplies intelligence.

The harness supplies structure.

Together they produce reliable execution.

If your agent keeps falling apart, stop adding adjectives to the prompt.

Build the environment it needs to succeed.

If You Made It This Far

Bookmark this guide.

Follow @LunarResearcher on X

Subscribe to my Substack

Send this article to someone who is still trying to fix every agent failure with a longer prompt.