Your Developers Are Using AI. Your Governance Layer Isn’t.

Your developers are already using AI.

Claude Code. GitHub Copilot. Cursor. A dozen other tools running in the background while they work. The productivity gains are real. Developers ship faster. Code review cycles shrink. Repetitive tasks disappear.

And nobody in your organization has meaningful visibility into any of it.

Last week we wrote about the AI governance gap — the pattern of fast adoption and slow governance we’ve watched play out with cloud and Kubernetes, now repeating with AI. And we wrote about what good AI agent governance looks like — the four layers enterprises need before autonomous agents go to production.

This post covers the other half of the problem. Not agents making autonomous decisions in the background — but the developers actively using AI tools every day, with no guardrails on what they send, what models they use, or what it costs.

The developer layer is where the governance gap is widest, and where it’s least visible.

The Questions Nobody Can Answer

We talk to engineering leaders regularly. The conversation usually starts with a budget review or a security audit. And it almost always surfaces the same four questions:

• Why did our AI spend double last month?
• Which developers are calling expensive models when a cheaper one would do?
• What data are our developers sending to these AI tools?
• Who approved the ML team’s $2,000 budget exception?

These are not exotic questions. They’re the same questions finance teams asked about cloud spend in 2013. The same questions security teams asked about SaaS tools in 2015. The same questions platform engineering teams asked about Kubernetes workloads in 2018.

And right now, most enterprises can’t answer any of them.

The Cost Problem

AI spending is growing faster than any enterprise budget process was designed to track. Anthropic recently confirmed a $30B annualized run rate — growing from $9B just six months prior. A recent analysis by Madrona found that the 1,000+ enterprise customers each spending $1M+ annually are driving almost entirely token-based revenue — every API call metered, every token counted. One developer tracked the equivalent of $15,000+ in API usage over eight months on what had been a flat-rate subscription. That arbitrage window is now closing.

Inside most enterprises today, AI spend is attributed at the organizational level — a single invoice from Anthropic or OpenAI, with no breakdown by team, product, feature, or developer. That’s the equivalent of getting one cloud bill with no line items. You know how much you spent. You have no idea where it went.

The specific cost governance problems are consistent across every engineering organization we talk to:

• No spend attribution below the org level — no per-team, per-developer, or per-feature visibility
• No model access policy — any developer can call the most expensive model for any task
• No session budget enforcement — a developer can burn through thousands of dollars in a single session with no alert
• No approval workflow — budget exceptions are informal, undocumented, and invisible to finance

The Security and Data Leakage Problem

The cost problem is visible, eventually. The data leakage problem often isn’t.

When a developer pastes a database schema into Claude to ask for help optimizing a query, no alert fires. No SIEM event triggers. No DLP rule catches it. The data leaves the building quietly, embedded in a prompt, and the only record is in Anthropic’s logs — not yours.

The Samsung incident — where engineers accidentally leaked proprietary source code via ChatGPT — was not an anomaly. It was an early public example of something that is now happening quietly at scale across every enterprise that has given developers access to AI tools without guardrails. Proprietary code. Internal API specifications. Customer data. Database schemas. Security configurations. All of it flowing through AI tools that the security team has never reviewed, in prompts that nobody is logging.

The challenge for security teams is structural. AI tools don’t look like traditional data exfiltration:

• No file transfer — data leaves in a prompt, not an attachment
• No unusual outbound connection — Claude Code and Copilot are authorized tools
• No SIEM alert — nothing in the traffic looks anomalous
• No DLP trigger — most DLP tools aren’t pattern-matching against LLM API calls

What Good Developer AI Governance Looks Like

The mental model is the same one that solved the cloud and Kubernetes governance problems: a policy layer that sits between developers and the tools they use, enforcing rules before the action happens — not logging it after.

For developer AI governance, that means four concrete capabilities:

Layer 1: Identity-Aware Model Access

Every AI call should be tied to a verified identity. Not a shared API key, not a team token — a specific developer, authenticated against your identity provider, with group membership that determines what they’re allowed to do.

This is the same principle you apply to any other privileged system. A developer in the ML team has access to frontier models. A developer in the frontend team doesn’t. A contractor has access to Haiku but not Sonnet. A security researcher can call any model but their sessions are audited.

Without identity-aware model access, you have no policy — you have a shared API key that anyone with a virtual key can use to call anything. That’s not a governance model. It’s the absence of one.

Layer 2: Cost Attribution and Budget Enforcement

Every token should be attributed to a developer, a team, and ideally a work item or ticket. Not after the invoice arrives — in real time, at the moment of the call.

Good cost governance means:

• Per-developer and per-team spend tracked in real time
• Session budgets enforced pre-call — not after the fact
• Model degradation when budget thresholds are hit — automatically route to a cheaper model, not a hard block
• Exception requests that go through an approval workflow, with a full audit trail of who approved what

The goal is not to restrict developers. It’s to make AI spend visible and attributable, the same way cloud spend became visible and attributable once FinOps practices matured. The organizations that get this right will have a structural cost advantage over the ones that don’t.

Layer 3: Prompt and Data Governance

This is the hardest layer to implement and the most important one from a security perspective.

Good prompt governance means knowing what data is leaving your environment in AI calls — and having a policy layer that can enforce rules on that content before it reaches the model. Not logging it after. Enforcing before.

In practice this means:

• Content scanning on prompts — PII detection, code classification, data sensitivity tagging
• Policy-based blocking or redaction — strip customer data from prompts before they go to the model
• Data residency enforcement — ensure calls containing certain data types only go to compliant providers
• Audit trail on prompt content — not just that a call was made, but what was in it

This is not about reading developer prompts. It’s about having a governance layer that enforces your data handling policies at the point where data leaves the building — the same principle as DLP, applied to the AI layer.

Layer 4: Audit Trail

Every AI call a developer makes should leave a record. Not just that it happened — who made it, what model they called, what it cost, what was in the prompt at a metadata level, and what policy authorized it.

This is the layer that makes everything else auditable. Without it, you can set all the policies you want — but you have no way to prove they were enforced, no way to investigate an incident, and no way to answer your auditor’s questions.

The audit trail for developer AI governance should be tamper-evident, queryable, and mappable to your compliance framework. When a regulator asks whether your developers are handling customer data appropriately in their AI workflows, your answer should be a structured report — not “we think so.”

The Governance Layer Nobody Has Built Yet

Here’s what makes developer AI governance different from agent governance: the surface area is larger, the volume is higher, and the developers using these tools have no idea governance is missing.

Your developers aren’t doing anything wrong. They’re using the tools that make them more productive. The governance gap isn’t their problem to solve — it’s the platform team’s problem to solve, with the right policy layer in place before the sprawl becomes impossible to manage.

We’ve watched this exact dynamic play out with cloud and Kubernetes. The teams that built the governance layer early — before the sprawl, before the audit finding, before the security incident — are the ones that came out ahead. The teams that waited paid a much higher remediation cost.

We built the policy engine that governs Kubernetes at scale in thousands of enterprises. The same problem is now happening one layer up — in every AI tool your developers use every day. We’ve done this before. We’re doing it again. Stay tuned!

For a technical perspective on governing AI agents at the infrastructure layer — from AIBOM attestation at build time to Kyverno admission policies at runtime — read: Shadow AI Is the New Shadow IT: Governing AI Agents from Code to Runtime.