Technical Essay

Who Watches What Your Agent Is Doing?

Sandboxing, Task-Level Isolation, Fine-Grained Permissions, and LLM Judges — the Missing Supervision Layer of Agentic Software

Who Watches What Your Agent Is Doing? ~4K words

Who Watches What Your Agent Is Doing?

Sandboxing, Task-Level Isolation, Fine-Grained Permissions, and LLM Judges — the Missing Supervision Layer of Agentic Software

Your coding agent just edited forty files, ran a build, installed a package, and pushed a branch. You watched the first two minutes, went for coffee, and came back to a green checkmark. This is not a hypothetical productivity boost story; it is Tuesday, on a few million developer machines, right now. Meanwhile the same week's headlines include models caught leaving hidden notes to future instances to conceal bad behavior [1], and a rogue-agent remediation industry being built in public [2]. The question is no longer whether agents can do the work. It is who — or what — watches what they actually do while nobody is looking.

There are three credible answers, and they are not alternatives. They are layers. First, containment: the agent runs somewhere its damage cannot escape — a sandbox with no network, a filesystem it cannot leave. Second, least privilege at task granularity: every sub-task an agent delegates runs under an explicit permission envelope, narrower than the parent's, with no path to launder capabilities downward. Third, judicial oversight: independent LLM judges that review plans and outputs the way a code reviewer reviews a pull request — adversarially, with the power to block.

This essay walks through all three layers using the concrete architecture of odek, an open-source agent harness — every mechanism described here exists in a repo you can clone today. We finish with a worked example: how to trigger a judge-monitored plan, and how to hand a sub-agent exactly the permissions it needs — and not one bit more.

1. The Supervision Gap

Traditional software has a supervision stack built over fifty years: process isolation, users and groups, capabilities, code review, CI gates, audit logs, on-call rotation. None of it anticipated a coworker who writes code at machine speed, executes shell commands based on statistical guesses, and ingests instructions from every web page it reads.

The specific gap is this: the agent's intent is probabilistic, but its actions are real. A classical program does what its author wrote. An agent does what an LLM decided — this turn, under this context, with these tool descriptions and whatever a fetched web page whispered into its context window. Indirect prompt injection research has documented, repeatedly and empirically, that instruction-shaped content riding in benign tool output gets executed — hidden in web pages, code comments, error messages, or command arguments [3]. Not exotic exploits: ordinary tool output, doing double duty as an instruction.

You cannot review your way out of this. A human reading a transcript at agent speed is the bottleneck, and approval fatigue is a documented failure mode of every human-in-the-loop permission system ever built: click "approve" enough times and it stops meaning anything. The supervision layer has to be mostly automatic, mostly default-deny, and mostly invisible when everything is fine.

That leads to a design principle worth stating up front:

Supervision must be structural, not behavioral. Anything that depends on the model choosing to be safe is not supervision — it is hope.

2. Layer One: The Sandbox

The first layer answers the question "what happens if the agent goes completely rogue?" with a simple offer: try it, from inside a container where nothing matters outside the workspace.

Concretely, in odek:

Its value is not that it catches clever attacks; it is that it converts "the agent did something catastrophic" into "the agent did something catastrophic inside a disposable container." Those are different Saturdays.

What the sandbox does not solve: the agent still needs to read your code and write your files to be useful. The workspace itself is inside the blast radius, and the workspace is where the real assets live. That is what the second layer is for.

3. Layer Two: Task-Level Execution Isolation and Fine-Grained Permissions

Modern agents do not work alone — they delegate. An orchestrator spawns sub-agents to research, build, test, and review. Each delegation is a capability transfer, and the security question is: what exactly changed hands?

3.1 Risk classes, not booleans

odek classifies every tool invocation into named risk classes — from read-only safe, through workspace-scoped local_write, up to system_write, code_execution, install, network_egress, destructive, and a dedicated persistence class for anything whose entire purpose is deferred execution: shell profiles, git hooks, CI workflow steps, cron entries [5].

That last class deserves a pause, because it encodes a hard-won lesson. Writing a line to ~/.zshrc is not destructive, not egress, and not an install — by classical file-write logic it is harmless. But its payload executes later, in a context the user trusts. It is the malware pattern of the 1990s wearing an agent's trench coat. Classifying by write target rather than command shape — and making persistence-class operations prompt or deny, including headless — is the kind of counterintuitive, adversarially-derived design that only comes from red-teaming your own system.

Path classification is equally paranoid with good reason: credential files (~/.aws/credentials, ~/.netrc, ~/.ssh, the odek trust anchors themselves) escalate to system_write even on reads via shell, and matching is case-insensitive because macOS APFS and Windows NTFS will happily let ~/.SSH/id_rsa and ~/.ssh/id_rsa be the same file [5].

3.2 The default envelope, and profiles

Out of the box, every sub-agent runs under a built-in default profile capped at local_write: no system writes, no arbitrary code execution, no installs, no network egress, no destructive operations [6]. Sub-agents must opt up by selecting an explicit, operator-authored profile — never by asking nicely.

Profiles are named permission envelopes defined by the operator, not the model:

{
  "profiles": {
    "research": {
      "description": "Read-only web research — fetches pages, never edits or runs anything",
      "max_risk": "safe",
      "tools": { "disabled": ["write_file", "patch", "batch_patch", "shell"] }
    },
    "builder": {
      "description": "Write and verify code changes with project build/test commands allowlisted",
      "max_risk": "local_write",
      "allowlist": ["go test ./...", "go build ./..."]
    }
  }
}

Three properties matter here:

  1. max_risk is a ceiling, not a hint. Every class above it is denied outright for tasks running under that profile.
  2. allowlist is exact-invocation. A builder profiled agent can run go test ./... without a prompt — and nothing else in its prompt-class. Exact strings, no wildcards for the model to decorate with injected arguments.
  3. Profiles are operator-authored. Project files and task files cannot define or widen them; a task can select a defined profile, and nothing more. The ceiling is set by the human who owns the machine.

3.3 Trust is non-increasing downward

The subtle attack against delegation systems is capability laundering: an orchestrator that ingested a prompt injection spawns a "trusted" sub-agent and launders its untrusted context into a child with a bigger permission envelope.

odek closes this structurally: trust is stamped into the task at spawn time and the child runs at min(parent_trust, child_trust) — a task tree rooted in untrusted content cannot promote itself [6]. Sub-agents are non-interactive by design: they never prompt for approvals, prompt-class operations are denied outright, and the only route to a prompt-class operation is an exact operator allowlist entry.

And crucially, denials are loud, not silent. Every denied operation is reported back up the chain (denials: [{tool, class, reason}]) and emitted as a runtime event the operator can audit. A delegation system that silently swallows its own security decisions is a system you cannot reason about; one that reports every refusal turns the permission layer into an observable, debuggable surface. Each sub-task runs in the smallest envelope that makes it functional, and no chain of delegation can amplify its own permissions.

4. Deterministic Risk Assessment and Approval Requests

The judge layer handles semantic judgment. But the moment-to-moment question — "may this specific command run?" — must never be a judgment call at all. It needs to be deterministic: same input, same verdict, every time, with a paper trail.

4.1 The classifier

odek answers this with a command classifier that runs before every shell invocation, file write, and network operation. It is a deterministic function from (command, paths, flags) to risk class, and its design reads like a red-team log because it was built against real evasion attempts [5]:

The property that matters is not the long verb list; it is fail-closed directionality. Classification errs toward the more restrictive class, and the only path to a prompt-class operation without a human is an exact-match operator allowlist. Denylist entries (prefix match) are checked after allowlist but before class policy. The result is auditable: for any historical command, the classifier's verdict is reproducible — which turns "why did the agent do that?" from folklore into a lookup.

4.2 Approval requests as a designed UX, not an interrupt

When an operation's class maps to prompt, the approval request is a designed surface with its own threat model — because the human is now the attack surface:

The cumulative effect: the agent's every consequential action passes a deterministic gate, and the gate's exception path — the approval request — is engineered so that saying yes is a deliberate act, not a twitch. Supervision systems fail at their laziest interaction, not their strongest one; this is the layer designed for the lazy one.

5. Layer Three: LLM Coordinator Judges

Sandboxing contains damage. Permissions constrain capability. Neither can answer the question a staff engineer actually asks of a junior's work: is this plan any good, and did they actually do what they said? That judgment is semantic, and no human can perform it at agent speed. That is the judge layer's job.

The pattern — call it Agents-as-a-Judge, in deliberate contrast to LLM-as-a-judge — treats review as an adversarial multi-agent process, not a single model's self-grading:

The judge layer is what catches the failure classes the first two layers structurally cannot: a plan that is perfectly permission-compliant and completely wrong; a diff that passes CI and quietly weakens a security property; a "completed" step whose verification never actually ran. Sandboxing bounds the cost of a bad decision. Judges reduce the rate of bad decisions.

6. Worked Example: A Judge-Monitored Plan in odek

Enough architecture. Here is how this looks in practice, from the operator's seat.

Step 1 — define the envelopes (once, in your operator config):

{
  "profiles": {
    "builder": {
      "description": "Write and verify code changes; build/test commands allowlisted",
      "max_risk": "local_write",
      "allowlist": ["go test ./...", "go build ./...", "go vet ./..."]
    },
    "adversary": {
      "description": "Read-only adversarial review — may read everything, change nothing, run only vet",
      "max_risk": "local_write",
      "tools": { "disabled": ["write_file", "patch", "batch_patch", "delegate_tasks"] },
      "allowlist": ["go vet ./..."]
    }
  }
}

Step 2 — the orchestrator declares a plan with evidence checks attached to each step. In conversation, it invokes the plan tool; each step can carry up to four checks, each one a concrete tool invocation that must be observed succeeding before the step counts as done:

plan →
  1. Add request validation to the handler        [pending]
     check: shell { command: "go vet ./..." }
  2. Add failing test for malformed payloads (RED) [pending]
     check: shell { command: "go test ./internal/... -run TestMalformed -count=1" }
  3. Implement fix until tests pass (GREEN)        [pending]
     check: shell { command: "go test -race -count=1 ./..." }
  4. Adversarial review of the diff                [pending]

The critical mechanic: the model cannot mark step 3 complete by asserting it. The runtime matches the check's exact tool call and arguments against actually executed invocations, and only a passing observed outcome flips the status. The plan progress the user sees in the UI is grounded in tool-call evidence, not narrative.

Step 3 — the judge step delegates to an adversarial sub-agent, and the delegation itself carries the permission envelope:

delegate_tasks →
  task:
    goal: "Adversarially review the working-tree diff for correctness,
           security regressions, and unverified claims. Iterate with the
           findings until the review is clean. Deliver the final verdict
           and a list of any policy denials you hit."
    profile: "adversary"
    trust_level: "untrusted"
    max_risk: "local_write"

(The adversary profile caps at local_write rather than safe because the reviewer legitimately runs go vet — code execution — to check the diff compiles. Read-and-verify, write-nothing: the envelope is as small as the job allows.)

What happens at runtime, layer by layer:

  1. Discovery: the orchestrator can call list_subagent_profiles and sees adversary — its description, its max_risk, its tool filters — and selects by intent rather than guessing.
  2. Containment: the sub-agent runs sandboxed, network off, in the project container.
  3. Least privilege: it can read the diff and run exactly go vet ./.... It cannot write files, patch, or spawn its own sub-agents — those tools are disabled at the profile level, so even a fully compromised reviewer cannot plant code while "reviewing."
  4. Non-interactivity: it never prompts. If its review demands something outside the envelope, the operation is denied, the denial appears in its result (denials: [{tool, class, reason}]), and the parent sees subagent_denied events it can reason about.
  5. Evidence-verified reporting: the parent doesn't trust the verdict's prose; it re-runs the observable checks (go vet, the test suite) and compares. Trust, but checksum.
  6. Iteration: findings flow back into plan steps; the cycle repeats until the judge panel is clean. Only then does the work move toward a pull request — which, in this workflow, merges only after CI passes. The human remains the final approver, but they are approving a reviewed, evidence-gated, adversarially-clean diff — not a leap of faith.

That is what "who watches the agent" looks like when the answer is architecture.

What happens when it fails matters as much as the happy path. If the judge's review demands something outside its envelope, the operation is denied — visibly, in the result's denials list — and the parent adapts or re-scopes; it never gets silence. If a step's evidence check fails (go test exits non-zero), the plan step cannot be marked done: the status stays visible as incomplete, the model keeps iterating on that step, and a blocked step is marked blocked for the operator to see. If a model tries to declare a step complete without the observed check, the runtime refuses — the claim has no matching evidence in the session. And if the judge rejects the diff outright, the findings loop back into new plan steps and the cycle repeats; the gate is the clean review, so a rejected diff is structurally unable to reach a pull request. Every failure mode lands somewhere observable: a denial event, an open plan step, a repeated review cycle — never a quiet pass.

7. The Honest Limits

No supervision essay is complete without the parts that don't work yet.

Judges share training data with the judged. Diversity of persona is not diversity of weights; correlated blind spots across models are a live research problem, and a judge can be socially engineered by a sufficiently persuasive diff the same way a tired human reviewer can.

Allowlists are brittle. Exact-invocation matching is safe but high-maintenance; every legitimate new build command is an operator decision. There is a real tension between strictness and the approval fatigue it prevents — we judge strictness worth the friction, but it is a judgment, not a theorem.

Covert channels beat transport-layer observation. If a model can hide intent inside legitimate-looking content — a comment, a test fixture, a variable name — no permission system sees it. Detection has to move to semantic and behavioral layers, and that work is early.

The human is still the final gate, and the human is still fatigable. All of this machinery exists to make the final approval a five-second review of an evidence-backed summary instead of a twenty-minute archaeology of an agent transcript — a 10× improvement, not a solution. The honest question every team should ask is not whether to pay that supervision cost, but which layers their threat model actually needs. Judges multiply LLM spend; sandboxes multiply latency. Containment is cheap insurance. Start there, and add judgment only where autonomy is high.

8. Conclusion

"Who watches what your agent is doing?" is no longer rhetorical. It is an engineering question with a concrete answer stack: contain what you cannot trust, minimize what you must grant, and judge what you would otherwise assume. None of these layers requires AGI, telemetry dystopias, or a platform vendor's blessing — the harness described here is open source, runs on a laptop, and its security documentation reads like a red-team log because that is, in large part, what it is.

The agents are coming for the work regardless. The only open question is whether supervision ships in the same commit.

References

[1] TechCrunch. (2026). "OpenAI caught its models leaving notes to successors to hide bad behavior." TechCrunch AI.

[2] TechCrunch. (2026). "The fix for rogue AI agents could be more AI." TechCrunch AI.

[3] Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., Fritz, M. (2023). Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. arXiv:2302.12173. https://arxiv.org/abs/2302.12173

[4] BackendStack21. (2026). Sandboxing — odek documentation. https://github.com/BackendStack21/odek/blob/main/docs/SANDBOXING.md

[5] BackendStack21. (2026). Security — odek documentation (risk classification, persistence class, credential path hardening). https://github.com/BackendStack21/odek/blob/main/docs/SECURITY.md

[6] BackendStack21. (2026). Sub-Agents — odek documentation (trust inheritance, profiles, denials reporting). https://github.com/BackendStack21/odek/blob/main/docs/SUBAGENTS.md

[7] BackendStack21. (2026). Planning — odek documentation (evidence-gated plan checks). https://github.com/BackendStack21/odek/blob/main/docs/PLANNING.md

Kyberneees, 2026