AI agent governance is governance applied to AI systems that act: deciding what agents may do and under whose authority, evaluating what they actually do, and keeping evidence of both. It extends AI governance from models and their outputs to actions and their effects.
Its purpose is to establish the authority, boundaries, policies, responsibilities and evidence needed to keep autonomous behavior accountable. The problem differs from governing a conventional application. In a conventional application, developers determine the operations it can perform and the order in which they occur. An AI agent selects and sequences its operations dynamically, often beyond any predetermined workflow, operates across several systems, and meets circumstances nobody anticipated when it was deployed. Governance therefore has to address two things at once: what an agent is authorized to do, and how its actual behavior can be examined against that authorization.
From AI governance to AI agent governance
AI governance already addresses whole systems, not only models: their purpose, data, evaluation, deployment, monitoring and oversight. Frameworks such as the NIST AI Risk Management Framework and ISO/IEC 42001 give that work an organizational shape. The EU AI Act attaches legal duties to some of it, and its record-keeping and human-oversight requirements apply to the high-risk AI systems it defines, not to every agent.
Agent governance extends that system-level governance to tool-mediated autonomous execution. Two features of agents drive the extension.
- Autonomy. The system chooses its own sequence of steps. What needs governing is a run that did not exist until the agent produced it.
- Tool-mediated execution. The output is no longer only text for a person to weigh. It is an operation on a real system. OWASP's guidance calls the resulting risk excessive agency: more functionality, permission or autonomy than the task requires.
Agent governance inherits everything existing AI governance asks for, and adds questions about authority, action and effect. It is related to, and distinct from, AI agent security, which defends agents and the systems they reach against misuse and attack. A well-secured agent can still do the wrong job.
Three responsibilities: compliance, completion, efficiency
It is tempting to treat governance as the question of whether an agent stayed within the rules. That is one responsibility of three.
Compliance
Did the agent act within its authority, its declared scope and the applicable policy? This is the most familiar responsibility and the one most tooling addresses.
Completion
Governance is not exhausted when every individual action complies. An agent can stay inside its permissions and scope while failing to finish, pursuing something irrelevant, repeatedly correcting its own mistakes, or reporting success without establishing that the requested outcome occurred. Four things are easy to confuse and should be kept apart.
| What is observed | What it establishes |
|---|---|
| A completed execution sequence | The agent stopped |
| The agent's claim of completion | The agent reported success |
| Verified achievement of the original objective | The requested outcome exists, confirmed from the systems concerned |
| Quality and completeness of the result | The outcome is good enough, by criteria someone set |
The first two establish that execution ended and that success was reported. The last two address verified achievement and acceptable quality. The original objective is the reference that makes those meaningful, and it should remain meaningful throughout the run, not only at the start.
Efficiency
Governance should also ask what successful completion cost. The relevant dimensions include execution attempts, retries and corrections, tool calls, model tokens, money, elapsed time, and human attention or intervention.
Cost per successful completion is usually more informative than cost per action or per model call, because it accounts for the retries, failures, corrections and human intervention that a per-call figure can obscure. An agent that reaches the right outcome after repeated unnecessary attempts raises a different concern from one that reaches it directly, even when both comply with every policy.
Efficiency is not the fewest possible steps. Re-running tests, checking a result against a second source, or asking a person before an irreversible change all add work, and are justified when they improve reliability, correctness or safety. The governance question is whether the work done was proportionate to the objective, by limits the organization has set.
Autonomy and drift
An autonomous agent has to be free to respond to what it learns during execution. A legitimate change in approach is not automatically drift. The challenge is distinguishing justified adaptation from a departure that no longer serves the original objective.
The relationship that governance needs to preserve runs like this:
> original objective → autonomous execution → new information → adaptation → continued accountability to the objective → verified completion
Both failure modes are real. Restrictions that are too rigid remove the autonomy that made an agent worth using, and rule out legitimate paths it would have found. Autonomy with no reference lets an agent wander, consuming resources on work nobody asked for. Governance does not settle that tension by formula. It keeps the objective in view, so that an adaptation can be recognized as one.
Authority, task, action, evaluation, evidence
One distinction does more work than any other: an agent having permission to access a system does not establish that every action it takes in that system is appropriate to its current objective. Governance has to keep five things separate.
- The authority granted to the agent. What it may reach, decided by whoever deployed it. See authorization and permissions.
- The objective and scope of the particular task. Usually far narrower than the authority. See declared intent.
- The operations the agent actually attempts.
- The policy and evaluation applicable to those attempts.
- The evidence available for later review. See governance evidence.
Two further distinctions run through all five. Declared authority is not verified authority: a record that an agent ran under a named owner is a claim until something checks it. And an attempted action is not a confirmed effect: knowing that a write was attempted is different from knowing what changed.
These relationships matter most where agents run long or wide. Over a long sequence, an early action changes what later ones mean, and a small departure compounds. Across several systems, no single system's log shows the whole run, and each system's permissions were granted without knowledge of the others. The thing that ties the run together is the task, which is why the task has to be stated and kept.
Governance across the life of a deployment
Before deployment. Define the agent's purpose and capabilities. Grant permissions deliberately. Assess risk, design policy, agree completion criteria and resource limits, and approve the deployment. Evaluation on representative tasks belongs here.
During execution. Runtime governance evaluates each attempted action against the declared intent, scope and policy as the agent runs, and records the result. That entry covers the subject in depth. Its place in the larger picture is to connect the authority decided beforehand with the review that follows.
After execution. Review runs, investigate incidents, verify outcomes against the systems acted on, improve policies and scopes, and retain evidence for as long as accountability requires.
Each stage produces something the next depends on. Policy written before deployment is what runtime evaluation applies. The record made at run time is what review reads. What review learns changes the policy.
Who is responsible
Technology does not hold accountability. People and organizations do, and agent governance has to name them.
- Who deploys the agent, and approved it for this use.
- Who establishes its authority, meaning the permissions and the systems in reach.
- Who owns its policies, and is entitled to change them.
- Who is accountable for its behavior when something goes wrong.
The organization or operator also sets the references that everything else is measured against: objectives, policies, acceptable risk, completion criteria and resource constraints. A governance system makes execution accountable to those references. It does not decide, for an organization, what level of risk or cost is acceptable for a given task.
Technical controls as complementary mechanisms
| Control | Its main contribution |
|---|---|
| Evals | Evidence, before deployment, of how an agent behaves on tested tasks, including whether it completes them |
| Guardrails | Checks that inputs, outputs or actions satisfy constraints defined in advance |
| Authorization | Limits on what the agent may reach |
| Gateways | Mediation of requests at the boundaries they control |
| Sandboxes | Limits on how far an action's effects can extend |
| Human oversight | A person's judgment at chosen points |
| Observability | Depth of insight into how a run unfolded, including its cost |
| Runtime governance | The relationship between each attempted action and the declared purpose, under policy, preserved as evidence |
No one of these governs an agent. Compliance draws mainly on authorization, guardrails and runtime governance. Completion draws on evals, on verification against the systems acted on, and on review. Efficiency draws on observability and cost data read against outcomes. Agent governance is often confused with one of its parts, most often with guardrails or observability, each of which is necessary and none of which is the whole.
An example
The following is an illustrative scenario, not a record.
A finance operations team deploys an agent to reconcile supplier invoices against purchase orders each month.
Before deployment. The finance systems lead approves the use. The agent is granted read access to the purchasing system and write access to reconciliation notes, and nothing else. The team writes a policy marking any payment operation as high-consequence. It sets completion criteria, that every invoice for the period is either matched or flagged with a reason, and a resource limit per monthly run. The controller is named as accountable.
During execution. The agent declares its objective and a scope covering the purchasing system. It works through the invoices. Midway it finds that forty invoices lack purchase order numbers and adapts, matching them by supplier and amount. That is a change of approach in service of the same objective. Later it attempts to email a supplier for a missing document. Email was never in scope, and the attempt is recorded as outside it.
At the end. The agent reports that reconciliation is complete. The team checks the purchasing system: three invoices are neither matched nor flagged. The execution sequence finished and the agent claimed success, and the objective was not fully achieved. Cost data shows that a third of the run's tokens went to retrying one failing lookup.
In review. The team decides the supplier email was a reasonable idea and widens the scope to permit drafting, not sending. It adds a retry limit. It treats the three missed invoices as a completion failure, not a compliance one, and changes how the agent confirms its own results.
Different parts of that story came from different places. The authority and criteria came from people. The record of what was declared and attempted came from runtime governance. Verification came from the purchasing system. Cost came from usage data. The judgments came from the review.
Where Sentience Governor fits
Sentience Governor is one implementation of the runtime governance and evidence portion of this larger problem. In its released 0.3.2 form it records a declared intent and its source, records each attempted action at the execution boundary where it is instrumented, evaluates those actions with deterministic rules against the declared scope, default policy and the operator's profile, and keeps the findings in a local record. It also attributes model tokens to the turns of a session and reports how much of that usage fell on turns that carried findings. It observes and flags, and every action proceeds.
That is a foundation for the three responsibilities, not the whole of any of them.
- It does not establish organizational accountability. The owner in its record is a claim, and it does not verify human authority.
- It does not enforce permissions.
- It does not establish verified task completion or judge the quality of a result.
- It performs no general semantic detection of drift.
- It does not optimize the economics of a task or judge whether a path was efficient. It reports token usage, not cost, and does not know whether a run succeeded.
- It sees only the systems it is instrumented on.
Where the concept stops
AI agent governance is an organizational discipline supported by technical mechanisms. No product supplies it, and a complete set of controls does not supply it either, without people who own the objectives, the policies and the consequences. Its test is practical: when an agent has finished, can the organization say what it was authorized to do, what it was asked to do, what it attempted, what that achieved, what it cost, and who answers for it?