Most enterprises we talk to are in the same place right now.
AI agents are running in development environments. Developers are using Claude Code, Copilot, and Cursor daily. Business units are asking when everything can go to production. And security teams are asking a different question entirely:
| How do we do this safely? |
It’s the right question. And the answer isn’t “slow down.” It’s “build the governance layer before you scale.”
Previously, we’ve covered the AI governance gap — the pattern we’ve watched play out with cloud and Kubernetes, now repeating with AI. We’ve written about what good AI agent governance looks like — the four layers every enterprise needs before autonomous agents go to production. And we’ve written about the developer governance gap — what it means when nobody owns the policy layer for the AI tools your developers use every day.
This post is about what comes next. The journey from where most enterprises are now — experimenting, moving fast, ignoring governance — to where they need to get to before something goes wrong.
Here’s what that journey actually looks like. And where most enterprises get it wrong.
Stage 1: Experimentation (Where Most Enterprises Are Now)
The experimentation stage feels low-risk. AI agents are running in sandboxes. Developers are using AI tools on approved projects. Nothing is touching production data at scale. Security teams are loosely aware but not actively engaged.
This is exactly when governance decisions get made — by accident.
The service account the developer used for the prototype becomes the service account the production agent inherits. The broad database permissions that made experimentation easy become the permissions nobody questions when the agent goes live. The AI tools that developers adopted without review become embedded in critical workflows before anyone has assessed the data they’re processing.
The governance layer doesn’t get built in experimentation because nobody thinks it’s needed yet. By the time it’s obviously needed, the patterns are already set. The cost of retrofitting is an order of magnitude higher than the cost of building it right the first time.
| The enterprises that get governance right don’t build it after the agents go to production. They build it before the first agent leaves the sandbox. |
What good looks like at this stage:
- Every agent registered with a verifiable identity — even in development
- Developer AI tool usage attributed to teams and individuals from day one
- Permissions scoped explicitly — not inherited from over-provisioned service accounts
- A policy template defined before the first workload touches production data
Stage 2: The Push to Production
The pressure to move from experimentation to production comes fast. And it comes from every direction simultaneously.
A competitor announces an AI-powered feature. A board member asks why the company isn’t moving faster. A business unit promises customers something the engineering team now has to deliver. The CFO wants to see ROI on the AI spend. The pressure is real, the timeline is short, and the path of least resistance is to ship and sort out governance later.
This is the most dangerous moment in the AI governance journey.
We watched this exact dynamic play out with Kubernetes. Teams that shipped without governance spent years retrofitting it. Security incidents, compliance findings, and audit failures that could have been prevented if the policy layer had been built before the workloads went live. The AI version of this mistake will be more expensive — because AI systems operate at machine speed and scale, across both the agent layer and the developer layer simultaneously.
| The Kubernetes lesson: the enterprises that waited to build governance until something went wrong paid a much higher remediation cost than the ones that built it early. AI will be no different. |
What good looks like at this stage:
- A governance gate before any agent or AI-powered feature touches production data
- Security sign-off on agent identity, permissions, and audit trail before go-live
- Developer AI tool usage governed — model access policies, budget enforcement, prompt guardrails in place
- A policy enforcement layer that can say no automatically — not flag and alert, but block
Stage 3: Governed Production
This is where enterprises want to be. It’s rarer than it should be.
Governed production doesn’t mean slow production. It means every AI system running in your environment — agents and developer tools alike — operates within a defined policy framework that your security team has approved and your compliance team can audit.
For agents, that means:
- A verified identity that travels with every request
- Explicitly approved permissions — scoped to what the agent actually needs
- A complete audit trail — every action logged, timestamped, and mappable to a compliance framework
- A policy engine enforcing all of the above at runtime
For developers, that means:
- Identity-aware model access — who can call which model, enforced before the call is made
- Spend attribution in real time — per developer, per team, per feature
- Prompt governance — a policy layer that enforces data handling rules before data leaves the building
- An audit trail that answers the CFO’s questions and the CISO’s questions simultaneously
The enterprises that reach governed production don’t get there by accident. They get there because somebody decided early — at the experimentation stage, before the pressure to ship arrived — that governance was a prerequisite for production, not an afterthought.
The Governance Checklist
Before any AI system — agent or developer tool — touches production, the minimum governance bar should include:
Agents
- Verifiable agent identity registered in a central registry
- Permissions explicitly defined and security-approved before go-live
- Every agent action logged, tamper-evident, and queryable
- Policy enforcement at runtime — violations blocked, not just flagged
- Policy exceptions require explicit approval and are auditable
Developer AI Tooling
- Model access tied to verified identity and group membership
- Per-developer and per-team spend tracked in real time
- Session budget enforcement before the call, not after the invoice
- Prompt content governance — data handling policies enforced at the API layer
- Full audit trail: who called what model, when, at what cost
A more detailed production readiness checklist will be published separately.
The Window Is Still Open
Most enterprises are still in Stage 1. That is genuinely good news.
The governance layer is far easier and far less expensive to build before agents go to production and before developer AI usage is fully embedded than after. The patterns are clear — we’ve seen them play out twice already, with cloud and with Kubernetes. The framework exists. What’s missing in most enterprises isn’t knowledge of what to build. It’s ownership of who builds it.
That’s the question we’ll address in the next post in this series.
| We’re working with a small number of enterprises on exactly this journey — building the AI governance layer before the pressure to ship makes it exponentially harder. If you’re in Stage 1 or Stage 2 and want to get governance right before you get to production, we’d like to talk. Reach out directly: ritesh@nirmata.com or DM on LinkedIn. |
For a technical deep-dive on governing AI agents at the infrastructure layer — AIBOM attestation, cosign signing, and Kyverno admission policies that block ungoverned agents before they ever run — read: Shadow AI Is the New Shadow IT: Governing AI Agents from Code to Runtime.
Next in this series: AI governance has no owner in most enterprises. That’s the real problem — and it’s more fixable than you think.
