Governance-as-Code for AI Agents

From Manual Oversight to Declarative Policy

As organizations deploy AI agents in production, a pattern emerges: governance starts as a manual process. Someone reviews agent logs daily. An engineer adds ad-hoc checks in application code. A Slack channel exists for “agent incidents.” This works for one agent running in a staging environment. It does not work for twenty agents running in production across multiple teams.

Governance-as-code is the practice of defining agent permissions, constraints, and compliance rules as version-controlled, declarative configuration. Instead of governance logic being scattered across application code, Slack runbooks, and tribal knowledge, it lives in YAML files and Rego policies that are reviewed, tested, versioned, and deployed through the same CI/CD pipelines as the rest of your infrastructure.

GovernorAI implements governance-as-code as a first-class concept. Every policy is a YAML document. Complex rules use Open Policy Agent (OPA) with Rego. Policies are versioned with full history. Rollback is a single API call. This post walks through the complete policy model.

Anatomy of a GovernorAI Policy

A GovernorAI policy is a YAML document that defines what an agent is allowed to do. Here is a complete policy for a finance agent in production:

id: finance-agent-policy
name: "Finance Agent - Production"
agent_id: "finance-agent-v1"
governance_mode: enforcement
fail_closed: true

session:
  max_steps: 50

cost:
  max_usd_per_session: 25.00

tools:
  allowed:
    - "erp.*"
    - "email.send"
  denied:
    - "shell.*"

rules:
  - id: high_value_payment
    priority: 1
    match:
      tool: "erp.process_payment"
      condition:
        field: "args.amount"
        operator: ">"
        value: 5000
    action: require_approval

Each field serves a specific purpose. Let’s walk through them.

id and name

The id is the unique, immutable identifier for this policy. It is used in API calls, audit logs, and version history. The name is a human-readable label displayed in the GovernorAI dashboard. Convention: use lowercase with hyphens for the id, and include the environment (staging, production) in the name for clarity.

agent_id

The agent_id binds this policy to a specific agent. When the agent with ID finance-agent-v1 submits a tool call to the GovernorAI gateway, this policy is selected for evaluation. One agent can have exactly one active policy. If you need different policies for different environments, use separate agent IDs (e.g., finance-agent-v1-staging).

governance_mode

The governance_mode determines how policy decisions are applied. GovernorAI supports three modes, designed for a progressive rollout path:

• audit_only — Every tool call is logged with the policy decision, but no actions are blocked. Use this when first deploying a policy to understand agent behavior without impacting functionality.
• shadow — Policies are fully evaluated and decisions are logged, but enforcement is not applied. The gateway returns “allow” for all actions while recording what the decision would have been. Use this to validate that a policy produces the expected decisions before enforcing it.
• enforcement — Full policy enforcement. Actions that violate policy are denied. Actions requiring approval are held pending. This is the production mode.

The recommended rollout path is: audit_only for initial observation, then shadow for decision validation, then enforcement for production. This progression lets you build confidence in policy behavior before any agent action is blocked.

fail_closed

When fail_closed is true, any action that does not match an explicit allow rule is denied. When false, unmatched actions are permitted. For production deployments, fail_closed: true is strongly recommended — it means an agent can only do what the policy explicitly permits, rather than everything the policy does not explicitly deny.

session

The session block defines per-session constraints. max_steps limits the total number of tool calls an agent can make in a single session. This prevents runaway loops where an agent repeatedly calls tools without converging on a result. When the step limit is reached, subsequent tool calls are denied with a clear reason.

cost

The cost block defines spending limits. max_usd_per_session caps the cumulative compute cost for a session. GovernorAI tracks cost per tool call and enforces the limit in real time. This prevents scenarios where an agent enters a loop that consumes expensive API calls — a common failure mode in autonomous agent systems.

tools

The tools block defines which tools an agent can and cannot call. allowed is a list of glob patterns that match permitted tool names. denied is a list of glob patterns that match prohibited tool names. Denied rules take precedence over allowed rules. The pattern erp.* matches any tool in the ERP namespace: erp.process_payment, erp.get_invoice, erp.list_vendors, etc.

rules

The rules block defines conditional logic for specific tool calls. Each rule has a match condition and an action. Rules are evaluated in priority order (lower numbers first). In the example above, the high_value_payment rule matches any call to erp.process_payment where args.amount exceeds 5,000, and requires human approval before the action proceeds.

OPA Integration for Complex Logic

YAML rules handle common patterns well — tool allowlists, argument thresholds, simple conditions. But real-world governance often requires more complex logic: time-of-day restrictions, cross-referencing external data, multi-field conditions, or organization-specific compliance rules.

GovernorAI integrates with Open Policy Agent (OPA) to support Rego-based policies for these cases. Rego is a purpose-built policy language that is declarative, testable, and widely adopted in infrastructure governance (Kubernetes admission control, Terraform plan validation, API authorization).

Here is a Rego policy that enforces business-hours restrictions and geographic compliance for payment processing:

package governor.finance

import future.keywords.if
import future.keywords.in

# Deny payments outside business hours (UTC)
deny[msg] if {
    input.tool == "erp.process_payment"
    hour := time.clock(time.now_ns())[0]
    not business_hours(hour)
    msg := sprintf(
        "Payment blocked: outside business hours (current hour: %d UTC)",
        [hour]
    )
}

business_hours(hour) if {
    hour >= 9
    hour < 17
}

# Deny payments to sanctioned countries
deny[msg] if {
    input.tool == "erp.process_payment"
    country := input.args.recipient_country
    country in sanctioned_countries
    msg := sprintf(
        "Payment blocked: recipient country %s is sanctioned",
        [country]
    )
}

sanctioned_countries := {"NK", "IR", "SY", "CU"}

# Require dual approval for cross-border payments over $10,000
require_approval[msg] if {
    input.tool == "erp.process_payment"
    input.args.amount > 10000
    input.args.recipient_country != input.args.sender_country
    msg := "Cross-border payment over $10,000 requires dual approval"
}

This Rego policy expresses logic that would be cumbersome in YAML: time-based conditions, set membership checks, multi-field comparisons. The policy is evaluated by OPA at runtime, and the results are combined with the YAML policy decisions. Rego policies can be unit-tested independently using OPA’s built-in test framework, which means governance rules get the same testing rigor as application code.

Policy Versioning and Rollback

Every policy change in GovernorAI creates a new version. The version history is immutable — previous versions are never modified or deleted. This provides a complete audit trail of policy changes over time, which is essential for compliance and incident investigation.

When a policy change causes unexpected behavior — agents being incorrectly blocked, or actions being incorrectly permitted — rollback is a single API call:

# Rollback finance-agent-policy to version 3
curl -X POST http://localhost:8081/api/v1/policies/finance-agent-policy/rollback \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"target_version": 3}'

The rollback creates a new version (version N+1) that contains the same policy content as the target version. This means the version history remains linear and append-only — you can always see that a rollback occurred, when it occurred, and who initiated it.

In practice, policy versioning integrates with your existing change management process. Policies live in version control alongside application code. Changes go through pull requests with review. CI pipelines validate policy syntax and run Rego unit tests. Deployment pushes the new policy to the GovernorAI control plane via API. If something goes wrong, rollback is immediate.

Progressive Rollout: A Practical Workflow

Here is the workflow we recommend for deploying a new policy or modifying an existing one:

Step 1: Write and test locally. Define the YAML policy and any associated Rego rules. Run OPA unit tests against the Rego policies. Review the policy in a pull request.

Step 2: Deploy in audit_only mode. Push the policy to GovernorAI with governance_mode: audit_only. All agent actions are logged with policy decisions, but nothing is blocked. Monitor the audit trail to understand how the policy would affect agent behavior.

Step 3: Promote to shadow mode. Update the policy to governance_mode: shadow. GovernorAI now fully evaluates every action and records what the decision would be, but still returns “allow” to the agent. Compare shadow decisions against actual agent behavior. Look for false positives (legitimate actions that would be blocked) and false negatives (problematic actions that would be allowed).

Step 4: Promote to enforcement. Once shadow mode confirms the policy behaves as expected, update to governance_mode: enforcement. The policy is now live. Monitor the audit trail and dashboard for the first 24-48 hours.

Step 5: Iterate. As agent capabilities evolve and business requirements change, update the policy through the same workflow. The version history provides a complete record of every change.

Governance-as-Code as Infrastructure

The principles behind governance-as-code are borrowed directly from infrastructure-as-code. Terraform taught us that infrastructure should be declarative, version-controlled, and reproducible. Kubernetes admission controllers taught us that policy enforcement should be deterministic and sit at the API boundary. OPA and Rego taught us that policy logic should be separated from application logic and independently testable.

GovernorAI applies these same principles to AI agent governance. Policies are declarative YAML, not imperative code. They are version-controlled with full history. They are deterministically evaluated at the execution boundary. Complex logic is expressed in Rego — a language designed for policy — and tested with OPA’s unit test framework.

The result is a governance model that scales with your agent fleet. One policy or a hundred policies, the workflow is the same: write, test, review, deploy, monitor, iterate. No manual log reviews. No ad-hoc checks in application code. No Slack channels for incident triage. Governance is code, and code is governed.