title: "Test What Your AI Agents Must Not Do"
slug: "test-what-your-ai-agents-must-not-do"
summary: "AI agent guardrails need negative tests. Ota refusal canaries verify that the real execution boundary still rejects selected unsafe tasks and workflows before any work starts."
publishedAt: "2026-08-12T13:00:00Z"
category: "field-note"
status: "published"
author: "bobai"
otaVersion: "1.6.25"
cover_image: "https://res.cloudinary.com/ota-run/image/upload/q_auto/test-what-your-ai-agents-must-not-do.png"

tags:

• aiagents
• agentsafety
• negativetesting
• executiongovernance

A Guardrail Without A Negative Test Is Still An Assumption

Most AI agent governance starts with the positive path.

A repository tells an agent which task it may run. The agent invokes that task. The test suite
passes. CI goes green.

That proves one useful thing: the allowed path still works.

It does not prove that the forbidden path is still forbidden.

A stale allowlist, an incomplete dependency closure, a runner regression, or a hand-maintained CI
condition can quietly widen what an agent is able to execute. Every positive check may remain green
while the safety boundary has stopped doing its job.

This is why agent guardrails need negative tests.

Ota calls those tests refusal canaries.

They are available in Ota v1.6.25, and they reflect a deliberate product position:

Agent safety should be executable contract truth, not a promise that the model will remember to
behave.

Ota uses the same ota.yaml contract to define the safe task surface, evaluate the complete
selected closure, refuse unsafe execution, emit machine-readable evidence, and project the canary
into CI. There is no separate shell rule for the negative test to drift away from.

That is the difference between documenting a guardrail and operating one.

What A Refusal Canary Tests

A refusal canary names a task or workflow that must remain outside the repository's agent-safe
execution boundary.

For example, a repository may allow agents to run verification while reserving publishing for a
human-controlled path:

agent:
  safe_tasks:
    - verify
  refusal_canaries:
    - task: publish
    - workflow: release

tasks:
  verify:
    command:
      exe: pnpm
      args: [test]
    safe_for_agent: true

  publish:
    command:
      exe: pnpm
      args: [publish]
    safe_for_agent: false

workflows:
  release:
    intent: release
    run:
      task: publish

The positive check proves that the allowed lane remains callable:

ota run verify --agent

The refusal canary tests the opposite boundary:

ota run publish --agent --expect-refusal
ota up --workflow release --agent --expect-refusal

These commands do not run the unsafe task and then inspect its exit code. Ota exercises the same
agent admission boundary used by ordinary execution and requires refusal before the selected task,
its dependencies, or the workflow closure starts.

That distinction matters. The canary tests the execution boundary, not the behavior of a dangerous
command after it has already begun.

Expected Refusal Becomes A Passing Test

An ordinary agent refusal is a blocked execution result. That is the correct behavior for a direct
attempt to run an unsafe task, but it is inconvenient as a CI assertion because the shell sees a
non-zero exit.

--expect-refusal gives that result explicit negative-control semantics:

• refused_as_expected means the declared agent-safety boundary refused the selected target before
  execution and the canary passes;
• refusal_not_observed means the target was admitted, so the canary fails; and
• wrong_refusal_boundary means execution was blocked for another reason, so the canary still
  fails.

The third state prevents an important false positive.

Suppose publish accidentally becomes agent-safe, but an unrelated organization policy still
denies the command. A weak negative test would see "something failed" and call the guardrail
healthy. Ota does not. The canary passes only when the safety boundary it was created to test is
the boundary that refused the lane.

For machine consumers, the same result is available as structured JSON:

ota run publish --agent --expect-refusal --json

The output carries the canary target, the runner-authored refusal record, whether execution
started, and the associated blocked receipt when refusal was observed. The agent does not
self-report that it behaved safely.

This is where Ota is intentionally more opinionated than a task runner. A task runner answers,
"Can I invoke this command?" Ota also asks, "Should this actor be admitted to this complete
execution closure, and what evidence should exist when the answer is no?"

Why This Must Run Through The Real Runner

It is easy to imitate a refusal canary with shell logic:

if [ "$AGENT_MODE" = "true" ]; then
  exit 1
fi

That only proves the shell condition still exists.

It does not prove that Ota's selected task closure, safety declaration, dependency expansion, and
execution admission agree. It can also drift independently from local execution, turning CI into a
second policy system.

A useful refusal canary has to pass through the same chokepoint as the real action. Otherwise the
test and the boundary can fail independently while both appear green.

Ota therefore derives the refusal from current contract truth at execution time. Authors declare
which task or workflow is the canary; they do not author the reason that should make it pass.

Put Refusal Canaries In CI

Local refusal gives agents and developers fast feedback. CI turns the same boundary into a
repository-level control.

A mature governance lane should test both directions:

1. At least one meaningful safe verification path is admitted and completes.
2. Each material unsafe path selected as a canary is refused by the agent boundary.

Ota's provider-neutral CI projection includes each declared refusal canary. The GitHub adapter
emits each one as its own provider check with a stable merge-check identity. That lets a repository
make the negative control merge-required instead of hiding several boundaries inside one opaque
script step.

This is stronger than one broad "agent safety" job. If the publish canary fails while verification
still passes, the provider shows exactly which execution boundary widened.

Branch protection remains provider-owned. Ota can generate and identify the GitHub check, but the
repository's GitHub policy must make it required if the organization wants a non-optional merge
gate. Future provider adapters must preserve the same ownership boundary rather than pretending
Ota controls provider policy.

What Refusal Canaries Prove

A passing refusal canary proves a narrow and valuable claim:

At this contract snapshot and runner boundary, the selected task or workflow was refused for the
expected agent-safety reason before execution started.

It does not prove:

• that every dangerous repository action has been declared;
• that the maintainer classified every task correctly;
• that raw shell execution outside Ota is impossible;
• that CI branch protection requires the canary check;
• that runtime filesystem, network, credential, or provider controls were enforced; or
• that the repository is globally safe for autonomous operation.

Those are different obligations.

Contract-claim assurance can evaluate whether observable repository evidence supports a declared
safety claim. Sandbox and capability providers can enforce runtime controls. CI policy can make
checks mandatory. Refusal canaries test whether one declared runner boundary still refuses what it
is supposed to refuse.

Keeping those claims separate is what makes the result trustworthy.

Turn Incidents Into Permanent Negative Tests

A refusal canary protects a boundary the repository has already identified. It cannot protect an
omitted path merely because that path reaches a similar outcome.

If publish is refused, Ota has proved that the selected publish closure was refused. It has not
proved that another task, migration, helper script, or raw shell command cannot reach the same
external effect. Presenting task-level refusal as effect-level protection would turn a precise
negative test into a broader claim the evidence does not support.

The practical operating model is a ratchet: when an incident or review exposes another material
execution path, add that real task or workflow as a permanent canary. The incident should improve
the executable governance suite, not remain only in a retrospective document.

Ota's current refusal canaries are task- and workflow-bound. Effect-level assurance across
equivalent execution paths is a separate, planned governance boundary; it is not something this
feature claims to provide today. Until that boundary is implemented and pressure-tested, canary
coverage should remain explicit about which declared lanes it exercises and which equivalent paths
remain unproved.

Choosing Useful Canaries

Do not turn every non-agent-safe helper into a canary. Choose actions whose accidental admission
would materially weaken the repository's governance story.

Good candidates include:

• package publishing and release workflows;
• production deployment;
• destructive database operations;
• credentialed external mutations;
• infrastructure apply or teardown tasks; and
• expensive or metered execution that should require a different authority path.

The canary should name the real contract task or workflow. Do not create a fake unsafe command
whose only purpose is to be refused; that proves the fixture, not the material repository boundary.

Also keep the positive lane. A system that refuses everything is not safely governed execution. It
is unavailable software. The useful result is selective admission: routine work remains easy, and
materially different work remains outside the agent-safe closure.

Try It In Your Repository

The fastest way to understand refusal canaries is to make one fail deliberately on a temporary
branch.

First, install Ota, then inspect the repository before changing its
contract:

ota doctor
ota tasks --use
ota tasks --safe --use

Add one real unsafe task or workflow to agent.refusal_canaries. Choose publishing, deployment,
infrastructure mutation, or another material lane that already exists in the repository. Then
validate both directions:

ota validate
ota run verify --agent
ota run publish --agent --expect-refusal --json

Now temporarily make publish agent-safe. The positive verification lane can remain green, but the
refusal canary changes to refusal_not_observed and exits non-zero. That is the failure shape the
feature exists to expose: the repository still works, but its agent execution boundary has widened.

When the local contract is correct, inspect the provider-neutral CI projection:

ota ci projection \
  --workflow verify \
  --mode native \
  --target-os linux \
  --json

For GitHub Actions, Ota can render the governed reusable workflow from that projection:

ota ci github render \
  --workflow verify \
  --mode native \
  --target-os linux

The repository keeps ownership of triggers, permissions, secrets, runners, environments, and
branch protection. Ota owns the contract-derived execution and refusal checks. That boundary lets
teams adopt Ota without replacing their CI provider or creating another hand-maintained workflow
language.

The Larger Point

AI governance cannot stop at telling agents what they may do.

It also needs tests for what they must not be allowed to execute.

Positive tests prove that an allowed path still works. Refusal canaries prove that a selected
forbidden path still stops at the runner boundary. Together they turn an agent allowlist from
static metadata into executable governance.

That is the standard Ota is building toward: boundaries that are declared once, exercised through
the real chokepoint, and retained as evidence rather than trusted as prose.

If your repository already tells agents what they may run, the next useful question is simple:

Which one unsafe action would you want CI to prove they still cannot run?

Go Deeper

• Safe Agent Execution and Refusal
• Contract-to-CI Governance
• Why Agent Safety Needs Enforced Boundaries, Not Just Declared Ones
• One Contract, Multiple Enforcement Points for Safe AI Agent Execution

Originally posted here: https://ota.run/blog/test-what-your-ai-agents-must-not-do

🔗 Nguồn bài viết gốc: Bobai Kato