Why AI Agents Need Governance Before Action

AI agents are moving from answering questions to taking actions.

That single shift changes the entire risk model.

Chatbots produce text. Agents operate systems.

They can read emails, call APIs, update customer records, deploy code, create tickets, approve workflow steps, coordinate with other agents, and store new knowledge in long-term memory.

That means the core question is no longer:

Is this answer correct?

It is:

Should this agent be allowed to act right now?

That is the question a governance layer exists to answer.

1. Memory is useful. It is not governance.

Memory helps agents preserve context across sessions.

It helps them remember user preferences, past work, tool outputs, and previous decisions. That matters. But memory does not decide:

• which facts are allowed to enter memory
• which memories are stale or poisoned
• which actions require approval
• which workflow step must come next
• which proof is required before execution
• which agent is allowed to use which information

In production agent systems, the harder problem is often not retrieval. It is authority.

Memory helps an agent remember.

Governance helps it choose what it is allowed to do.

This distinction is where Marrow sits.

Marrow is not just a memory layer. It is a judgment layer for AI agent fleets.

2. Observability explains the past. Agents need control before the future.

Observability is necessary. Teams need traces, logs, evaluations, annotations, alerts, and dashboards.

But post-hoc visibility does not stop a bad action before it happens.

If an agent deploys unsafe code, sends a sensitive email, approves the wrong payment, or skips a required workflow step, the dashboard may explain the incident later. It does not necessarily prevent it.

For high-impact workflows, the signal has to reach the agent before action:

• risk level
• required proof
• policy constraints
• owner approval
• rollback plan
• exact next step
• allow, warn, review, or block

This is the gap between observability and governance.

Dashboards inform humans.

Governance informs agents before they act.

3. Evaluation is a snapshot. Production is a stream.

Pre-deployment evaluation catches many failures. But production agents live in changing environments.

Prompts change. Tools change. APIs change. Data changes. Policies change. Other agents in the fleet change.

A benchmark may say an agent performs well under one set of conditions. Production asks a different question:

Will this agent keep making the right decision as the environment changes?

Research on tool-using agents points in the same direction.

ToolEmu studies language-model agents using high-stakes tools and shows that agent failures can create severe real-world consequences. AgentHarm and CUAHarm focus on harmful behavior once agents can use tools or operate computers. Other research shows that agents may understand risk in the abstract but still fail to avoid risky actions in concrete trajectories.

The lesson is practical:

Safety cannot live only in the model or only in a benchmark.

Production agents need runtime control.

4. Governance is becoming AI infrastructure.

Major AI governance frameworks are converging on the same idea: responsible AI has to become operational.

Not principles alone.

Not policy PDFs alone.

Not dashboards alone.

Operational governance requires:

1. documented risk management
2. automatic logging
3. human oversight for high-impact decisions
4. quality management across the lifecycle
5. traceability of system behavior
6. post-market monitoring
7. accountability for adverse outcomes

This is visible across NIST AI RMF, NIST's Generative AI Profile, ISO/IEC 42001, the OECD AI Principles, the EU AI Act, OWASP's agentic AI risk taxonomy, and the 2026 Five Eyes guidance on agentic AI services.

The direction is clear.

Enterprise AI systems need evidence, control, traceability, and accountability across their operating life.

For AI agents, that means governance has to move into the runtime.

5. What an agent governance layer should do

A governance layer is a control plane between the agent runtime and the systems the agent can affect.

It receives the agent's intended action, evaluates it against policy, permission, risk, proof, and prior outcomes, then returns an enforceable decision:

allow

warn

review required

block

A serious governance layer needs nine functions.

1. Identity and scoped authority

Every agent needs a clear identity, limited permissions, and scoped credentials.

A fleet cannot be governed if every agent shares the same API key.

1. Runtime policy

Policies must become executable runtime conditions.

Which step comes first?

Which proof is required?

Which action always needs review?

Which action should never run automatically?

1. Risk gates

The system should classify actions by impact, reversibility, sensitivity, and business context.

Low-risk work may run automatically. High-risk work may require proof, approval, or blocking.

1. Proof packs

Before action, the agent should attach evidence.

Examples:

• test coverage
• rollback plan
• policy clause
• identity verification
• clinical approval
• bill review
• supervisor sign-off

1. Approval routing

Human review should not be a manual queue for everything.

It should be a conditional checkpoint for actions where the cost of error is high.

1. Audit and provenance

Every decision should be traceable.

Who initiated it?

Why was it allowed or blocked?

Which policy applied?

Which proof was attached?

Who approved it?

What happened next?

1. Outcome closure

Governance does not end at allow or block.

The system must close the loop:

• did the action succeed?
• did it fail?
• was rollback needed?
• who approved it?
• what lesson should be stored?

1. Memory write governance

Not every log is knowledge.

Not every piece of knowledge should influence every agent.

Real outcomes should become memory only through controlled writing.

1. Fleet-level learning

The governance layer should track behavior across the fleet:

• drift from baseline workflows
• retry loops
• repeated failures
• patterns that should warn future agents

This is where governance becomes more than restriction. It becomes learning.

6. Marrow as a governance case study

Marrow is designed around a simple operating loop:

orient−>think−>act−>check−>commitorient -> think -> act -> check -> commit

Each step has a governance function.

Orient surfaces relevant history, warnings, and constraints.

Think evaluates the intended action before it happens.

Act executes with context and guardrails attached.

Check inspects whether proof or closure is missing.

Commit records the outcome so the next decision improves.

This is the core difference between memory and judgment.

Memory asks:

What does the agent know?

Marrow asks:

Should the agent act, and under what conditions?

Its product surfaces map directly to runtime governance:

• decisionBrief() gives the agent pre-action context.
• workflowGate() returns allow, warn, review required, or block.
• runGuarded() wraps risky work with pre-action guidance and outcome closure.
• agentRuntime() injects lessons and proof requirements into the agent context.
• agentStatus() shows whether Marrow is active and collecting useful signal.
• valueReport() turns governance into owner-visible proof.

In other words:

Marrow turns past outcomes into before-action judgment.

7. A concrete example: the bad deploy

Imagine a CI/CD agent about to deploy a payment webhook change.

The build passes.

The branch is ready.

The agent is about to push to production.

But the pull request is missing three things:

1. test coverage
2. rollback plan
3. smoke test

Without governance, the deploy may go through.

The payment failures appear hours later. Customers are affected. The on-call engineer rolls back manually. The incident becomes another postmortem.

With governance, the workflow gate runs before deploy.

It returns:

risk level: high

proof pack missing

deployment blocked

The agent is not just logging more. It is being forced to stop at the right point.

That is the value of pre-action governance.

8. The right metrics for agent governance

A governance layer should not only count logs.

It should measure whether the fleet is becoming safer, more consistent, and easier to audit.

Useful metrics include:

• Action coverage: how many high-impact actions pass through a gate
• Proof completion rate: how many actions include required proof before execution
• Outcome closure rate: how many decisions are closed with a real result
• Prevented repeat failures: how often known failure patterns are avoided
• Drift severity: how far agents move from approved workflows
• False positive rate: how often the gate blocks too much
• False negative rate: how often dangerous actions slip through
• Audit reconstruction time: how long it takes to explain a decision
• Human review precision: whether human approval is used for the right actions

Governance is useful when it reduces rollback, prevents repeated incidents, narrows unnecessary review, and makes agent behavior easier to prove.

Otherwise it becomes compliance theater.

9. The governance layer has risks too

A governance layer can also fail.

It can be misconfigured. Policies can become stale. Proof can be incomplete. Gates can overblock. Attackers can target the control plane. If it stores too much sensitive data, it becomes a risk concentration point.

So the governance layer itself needs discipline:

1. Least privilege by default Agents should only receive the permissions they need, for the time they need them.
2. Fail closed for high-impact actions If the action can cause serious harm, missing proof should stop execution.
3. Fail soft for low-risk automation Not every action deserves the same level of friction.
4. Human-readable evidence, machine-readable policy Operators need to understand the decision. Systems need to enforce it.
5. Learning from real outcomes Without outcome closure, the system accumulates logs, not judgment.

10. Conclusion

AI agents push software into a new operating model.

They can choose tools, coordinate with other agents, modify systems, and create consequences.

In that model, governance is not a layer added after deployment. It belongs inside the runtime.

Memory, observability, evaluation, and human review all matter. But each solves only part of the problem.

A governance layer connects them into an accountable action loop:

policy before action proof before execution authority during action outcome after action learning across the fleet

That is the thesis behind Marrow.

AI agent fleets do not simply need to remember more.

They need better judgment before they act.

References

1. NIST. AI Risk Management Framework
2. NIST. Generative AI Profile, NIST AI 600-1
3. OECD. OECD AI Principles
4. European Commission AI Act Service Desk. Article 9: Risk management system
5. European Commission AI Act Service Desk. Article 12: Record-keeping
6. European Commission AI Act Service Desk. Article 14: Human oversight
7. European Commission AI Act Service Desk. Article 17: Quality management system