Skip to content

All posts

AI

How to audit what your AI coding agent did

Ritesh Patel

AI: How to audit what your AI coding agent did

To audit an AI coding agent, log each tool call with the identity and session behind it, and log refusals too. In Claude Code, the OpenTelemetry logs exporter writes a tool_decision event for every permission decision and a tool_result event for each call that runs. Hooks can keep a second copy on the developer’s machine.

Checked against the Claude Code monitoring and hooks docs, fetched 2026-10-29. Those pages name no single Claude Code version for this setup. Where a field needs a newer release, the post says so.

What an audit record has to answer

An audit of agent activity has to answer four questions: who made the call, what the call was, whether it was allowed and by which decision, and what came back. No single Claude Code event answers all four. The table maps each question to the event and field that carries it.

Question Where Claude Code records it Fields
Who acted Standard attributes on every event user.email, user.account_uuid, user.id, session.id
What the call was tool_decision, tool_result tool_name, tool_use_id, tool_parameters with detail logging on
Allowed or refused, and by what tool_decision decision, source
What came back tool_result success, error_type, duration_ms
Which prompt caused it Every event prompt.id

Identity depends on how the developer signs in. The gotchas section below covers the cases where user.email is missing.

This post covers the general Claude Code audit path. The MCP-specific setup and fields are in how to audit MCP tool calls.

Step 1: Turn on the logs exporter

Set up the logs exporter as the first step of how to audit MCP tool calls describes. That post has the exporter block, the managed-settings rule, and the check for claude_code.user_prompt. The events this post reads come from the same setup.

Repository settings cannot set the exporter variables. Claude Code ignores the OpenTelemetry exporter variables in a repository’s .claude/settings.json and .claude/settings.local.json, so a repository cannot choose the collector or turn on content capture. A repository can still set an exporter selector to none, unless managed settings or the launch environment sets it.

Step 2: Read the decision and result events

The two events answer different questions. tool_decision records every decision, including refusals. tool_result records only calls that ran. Join them on tool_use_id.

tool_decision tool_result
When it is emitted For every permission decision, accept or reject Only after the tool finishes running
Refused calls Yes, with decision set to reject No
Outcome Not recorded success is "true" or "false", plus error_type on failure
Where the decision came from source: config, hook, user_permanent, user_temporary, user_abort or user_reject decision_source: config, hook, user_permanent or user_temporary
Command or arguments tool_parameters, when detail logging is on tool_parameters and tool_input, when detail logging is on

Step 3: Add a local hook log

The collector is the record you rely on. A hook log gives a second copy on the developer’s machine. PostToolUse runs after a tool call succeeds, and PostToolUseFailure runs after one fails. Both match on tool name, and a matcher of "*" covers every tool.

  1. Create the directory ~/.claude/audit first. The command below appends to a file in that directory and does not create it.
  2. Add the configuration below to ~/.claude/settings.json, or to managed settings to keep it in place for every developer.
  3. Make one tool call and check that ~/.claude/audit/tool-calls.jsonl gained a line. The command needs jq on the PATH.
{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "*",
        "hooks": [
          {
            "type": "command",
            "command": "jq -c '{ts: (now|todate), event: .hook_event_name, session: .session_id, tool: .tool_name, tool_use_id: .tool_use_id, agent: (.agent_id // null)}' >> \"$HOME/.claude/audit/tool-calls.jsonl\""
          }
        ]
      }
    ],
    "PostToolUseFailure": [
      {
        "matcher": "*",
        "hooks": [
          {
            "type": "command",
            "command": "jq -c '{ts: (now|todate), event: .hook_event_name, session: .session_id, tool: .tool_name, tool_use_id: .tool_use_id, error: .error, agent: (.agent_id // null)}' >> \"$HOME/.claude/audit/tool-calls.jsonl\""
          }
        ]
      }
    ]
  }
}

The hook writes the event, session, tool, tool_use_id and subagent ID. It leaves out tool_input, because a Bash command or a file body can hold a secret. Hooks from settings files also run inside subagents, and the input carries agent_id when a subagent makes the call.

Where the record has gaps

Each path leaves a different trace. Check this table against the controls you plan to rely on.

Case What records it
Allowed by a rule, ran and succeeded tool_decision with decision set to accept, tool_result with success set to "true", and the PostToolUse hook
Refused by a deny rule, or by the user at a prompt tool_decision with decision set to reject. No tool_result is emitted, and no PostToolUse or PostToolUseFailure hook runs
Ran and failed tool_result with success set to "false", and the PostToolUseFailure hook
Denied by auto mode tool_decision. The PermissionDenied hook fires. A safety_event with category auto_mode_block or auto_mode_no_verdict is normally emitted. Some refused calls have none.
A file changed by a Bash command The Bash tool_result. A hook matched to Edit|Write does not run for that change

Claude Code emits the raw event stream only. Alerting, baselines and correlation across sessions belong in your SIEM or observability backend.

Gotchas before you rely on this

  • Identity depends on sign-in. With a direct API key, or on Amazon Bedrock, Google Cloud’s Agent Platform or Microsoft Foundry, only user.id and session.id are populated. Attach a user with OTEL_RESOURCE_ATTRIBUTES, set per user through managed settings or a launch wrapper.
  • Local transcripts are swept. Claude Code keeps session transcripts under ~/.claude/projects/, and the retention period is set by cleanupPeriodDays in managed settings. A machine where nobody starts Claude Code does not run the sweep, so a missing sweep event is not proof of retention.
  • Some fields need newer releases. request_id on tool_decision and tool_result needs Claude Code v2.1.310 or later. It links a call to the API response that produced it. On older releases, join on tool_use_id alone.

Decisions to make before rollout

  1. Which collector may hold prompts, commands and tool input, and whether it is approved to do so.
  2. Whether OTEL_LOG_TOOL_DETAILS stays on for every developer, or only for the teams that need command-level detail.
  3. Who reviews tool_decision events with decision set to reject, and how often.
  4. How long the collector keeps events, and what cleanupPeriodDays is set to on developer laptops.
  5. Whether the local hook log is kept at all, or whether the collector is the only record.

Enforcing this with AIControls

The audit record shows what an agent did after the fact. AIControls covers Claude Code with three controls: the gateway for Claude Code (CLI and VS Code), per-user tokens, and per-user and per-session cost and budget limits.

The gateway does not replace the record above. Keep the collector and the hook log as the record you review. For the deny-rule and hook side of tool control, see how to block destructive commands in Claude Code. For the wider approach, see governance for AI coding agents, and for the product, see AIControls.

Frequently asked questions

Does a file change made through Bash show up as an Edit or Write?

No. The call is recorded as a Bash call. Its tool_result names the tool Bash, and the command appears in tool_parameters when OTEL_LOG_TOOL_DETAILS is set to 1. Claude Code does not run a PostToolUse hook matching Edit|Write when a Bash command rewrites the same file, so a hook matched only to those tools misses the change. The hook in this post uses the matcher "*", which covers Bash.

Does the audit capture the Bash commands and file contents an agent used?

Not by default. Tool parameters, including Bash commands and tool input, are logged only when OTEL_LOG_TOOL_DETAILS is set to 1. Prompt text is redacted unless OTEL_LOG_USER_PROMPTS is set to 1. Turn these on only when the collector is approved to hold the same sensitive content the session contains. Otherwise the event stream records that a call happened, but not what it contained.

Can a developer switch off the audit on their own machine?

They can, unless the setting is in managed settings. A developer's shell or user settings can set OTEL_LOGS_EXPORTER to none, and a repository's settings file can do the same unless a managed setting or the launch environment sets that variable. Put the exporter selectors in managed settings to lock them. Keep the collector as the main record. A hook log lives in a file the developer can edit, so treat it as a local copy.

Which field joins a hook record to a telemetry event?

tool_use_id. Claude Code sets it on tool_decision and tool_result events and passes the same value to PreToolUse, PostToolUse and PostToolUseFailure hooks. Join the two on that field. Every event from one user prompt also shares a prompt.id, and hook input carries the same value as prompt_id, so you can group a hook record by prompt as well.

Ritesh Patel, Founder & VP Product, Nirmata

Ritesh Patel is a founder of Nirmata and leads its products. He writes about governing AI coding agents, MCP security and Kubernetes policy as code.

LinkedIn

Govern AI agents with AIControlsControl what AI agents do, and prove it.
ShareXLinkedIn

Stay up-to-date with the latest in AI, governance, and infrastructure

Subscribe to our free monthly newsletter to get industry news, DevOps tips, and articles right to your inbox on Kyverno, containerization, Kubernetes deployment & management and more.