Over the past year, nearly every engineering org I talk to has reached the same milestone: AI agents are no longer a demo. They’re calling real tools, against real systems, with real consequences. And nearly every one of those orgs has reached for the same first fix — an AI gateway.
That instinct is right. What’s often wrong is the assumption about what the gateway is actually doing for you.
The myth: visibility is governance
Here’s the belief I keep running into, stated plainly:
“I can route between models, I can see which tools my agents are calling, so I have this under control.”
It’s an understandable belief. A wave of vendors has emerged over the past eighteen months offering exactly this — model routing for cost optimization, dashboards showing which MCP servers are active, usage reports broken out by team. All useful. None of it is governance.
Routing and visibility answer the question “what happened?” Governance answers a harder question:
“was this agent allowed to do that, and can we stop it before it happens again?”
Those are different problems, solved by different architecture, and conflating them is the single most common mistake I see teams make when they evaluate an AI gateway.
Routing is an actuator, not a control plane
Start with routing. A model router decides which LLM handles a given request — maybe Claude for complex reasoning, a smaller model for simple lookups, a fallback if a provider is down. This is a real and valuable capability. It optimizes cost and latency.
But rerouting a request to a different model has nothing to do with whether the action that request triggers should be allowed. If an agent is about to delete a database record, it doesn’t matter which model decided to call that tool — what matters is whether the call itself should be permitted, for this agent, with these arguments, right now. A router has no opinion on that question. It wasn’t built to.
Treating routing infrastructure as a governance layer is like treating a network load balancer as a firewall. They sit in a similar part of the stack. They do fundamentally different jobs.
Seeing a tool call isn’t the same as controlling one
The second half of the myth is subtler, and it’s the one worth spending the most time on, because it’s where most of the current generation of “AI security” tooling actually lives.
Many gateways and network-monitoring tools can tell you that an agent called a tool — execute_query was invoked, send_email fired, a request went out to a particular MCP server. This is presented as governance. It isn’t. Knowing that a query ran is not the same as knowing what the query was. Knowing an email was sent is not the same as knowing what it said or who it went to.
Architecturally, this distinction comes down to where a tool sits relative to the traffic. Tools that watch network traffic from the outside — intercepting HTTPS connections, logging that a call to an AI provider occurred — see the envelope, not the contents. They can tell you an agent talked to a database. They generally cannot tell you, and certainly cannot block, the specific SQL statement inside that call.
Real governance requires sitting inside the protocol conversation itself — seeing the tool name, the exact arguments, and the response, before any of it executes. That’s the difference between an audit log and an enforcement point. One tells you what already happened. The other decides what’s allowed to happen next.
Having an MCP gateway doesn’t mean you’re covered
MCP has become the default frame for “agent governance” over the past year, which makes the next version of the myth an easy one to fall into: “I’ve got an MCP gateway in front of my agents, so I’m covered.”
Two things that assumption misses, and both are structural, not edge cases.
First, an MCP-only gateway is completely blind to everything that isn’t an MCP call. Most agent stacks still make plenty of direct LLM API calls that never touch an MCP tool at all — a raw completion request, a call out to a hosted model for classification or summarization, anything that isn’t wrapped in a tool invocation. If your governance layer only inspects MCP traffic, none of that is visible to it. You can have airtight MCP tool-call policy and still have no real idea what’s going into or coming out of your actual model calls.
Second, agents don’t reliably stay inside MCP even for the actions MCP was meant to cover. An agent with shell access can run a local command directly instead of calling an MCP filesystem tool. An agent with any code-execution capability can make a raw HTTP request to an internal API instead of going through a registered MCP server. Neither of these has to be malicious, or even unusual — it’s often just the shortest path the model found to get something done. If your only enforcement point sits at the MCP layer, both of these paths walk straight around it, in plain sight.
Put together: an MCP gateway alone typically governs one of three surfaces an agent actually operates on. LLM calls, MCP tool calls, and direct HTTP or local execution are three different traffic types, and a governance layer watching only one of them has blind spots by construction — not by mistake, and not something a better MCP policy fixes.
The fix isn’t a more sophisticated MCP gateway. It’s a single enforcement layer that spans all three — LLM, MCP, and HTTP — so the same identity, policy, and audit trail apply no matter which path the agent actually takes.
Breadth is not depth
There’s a third version of the same myth, and it’s arguably the most persuasive one, because it looks like thoroughness rather than a shortcut: “we integrate with everything.” A long list of connectors — Slack, Salesforce, GitHub, thirty other logos — reads as comprehensive coverage. It’s an easy thing to put on a slide and an easy thing to win a feature checklist against.
But breadth of integration and depth of enforcement answer two different questions. Integration breadth tells you how many places a tool can plug in. Enforcement depth tells you what happens once it’s plugged in — whether it can actually inspect and stop a specific action, or whether it just confirms the connection exists.
This gap is most visible in the environment where the most consequential agents actually run today: Kubernetes. A wave of vendors will happily show you a dashboard connecting to dozens of SaaS tools, but ask the same vendor how they govern an agent running as a workload inside a K8s cluster — with its own service-account identity, calling internal tools, making egress calls to systems well beyond a tidy SaaS API — and the answer usually gets much thinner, much faster. Broad, shallow SaaS coverage and deep, Kubernetes-native enforcement require genuinely different architecture. Few vendors have invested seriously in both, and the ones leading with breadth are rarely the ones who’ve done the harder work underneath.
If governance is the actual goal, breadth of connectors is a distribution question. Depth of enforcement — especially in the environment where your most autonomous, highest-risk agents are running — is the security question. Don’t let a good answer to one stand in for the other.
A buyer’s checklist: twelve questions to ask any vendor
If you’re evaluating gateways, routers, or “AI security” platforms right now, here are twelve questions worth asking directly. They apply to any vendor you’re looking at — including us.
1. Does it evaluate the actual arguments of a tool call, or just the tool’s name? “Agent called a database tool” is not the same as “agent tried to run DROP TABLE.” Ask to see a policy that inspects arguments, not just endpoints.
2. Does enforcement happen before execution, or only in a log afterward? If the honest answer is “we’ll show you what happened in the dashboard,” that’s observability. Governance means the action can be stopped before it runs.
3. Is every decision bound to an identity — the agent, and the human who delegated to it? An agent acting on someone’s behalf should carry that person’s identity and permissions through the whole chain, the same way you’d expect from any other privileged system.
4. Is there a real human-in-the-loop path for high-risk actions, or just an alert? An alert after the fact doesn’t stop anything. A genuine approval workflow pauses the action and waits for a human decision.
5. Is the audit trail immutable and mapped to a compliance framework, or a best-effort log? When your security team or an auditor asks “prove this didn’t happen,” a vendor should have a clear, exportable answer — not a promise that the logs are probably still there.
6. Is policy written and reviewed like code, or configured through a vendor UI you can’t diff or roll back? Policy-as-code, versioned in Git, reviewed in pull requests, is the difference between governance you can audit and governance you have to trust blindly.
7. Does inspection happen locally, or does your data leave your environment to be scanned? Some platforms process prompt and tool-call content on their own infrastructure to check for PII or policy violations — which means your data left your boundary to get governed. Ask where the scanning actually runs.
8. Does policy travel with the agent across model providers, or is it tied to one gateway? If switching from one LLM provider to another — or adding a second — means rebuilding your governance rules from scratch, what you have is provider-specific configuration, not a governance layer.
9. Does it notice when an agent’s behavior changes, or only enforce the policy you configured on day one? An agent that starts touching new tools or new data than it did last month is often the first sign of a problem — or a legitimate change that policy hasn’t caught up to. Static, “set once” policy misses this entirely.
10. Can it run inside your own infrastructure, or does it require routing traffic through a vendor-hosted cloud? For regulated or security-conscious teams, whether the control plane lives in your own environment or a third party’s SaaS is often the actual deciding factor — worth confirming before, not after, a security review.
11. Does it enforce with the same depth in Kubernetes as it does in its SaaS integrations? A long list of connectors is easy to build. Enforcing policy on an agent running as a workload inside your own cluster — with its own service-account identity, calling internal tools — is a different and harder problem. Ask to see it work there specifically, not just against a SaaS API.
12. Does the same policy and audit trail apply across LLM calls, MCP tool calls, and direct HTTP or local execution — or only to MCP? An MCP-only gateway is blind to raw LLM API calls and to anything an agent does by shelling out or calling an internal API directly. Ask specifically what happens when an agent bypasses MCP entirely — if the honest answer is “we wouldn’t see that,” the coverage gap is real, not hypothetical.
If a vendor can’t answer more than three or four of these with a concrete yes, what you’re buying is visibility with a governance label on it.
Where the industry is heading
It’s worth noting this isn’t just our own framing. The broader AI governance category — including how major analyst firms are now defining the space — increasingly draws a hard line between platforms that merely observe AI activity and those that enforce policy at runtime. The consistent finding across that research: enforcement at runtime is the capability most vendors claim and fewest actually deliver. Dashboards are easy. Blocking a bad tool call before it executes is not.
Where that leaves buyers today
To be fair to the current market: most available tools genuinely are visibility-first, and that’s not a criticism of any one vendor so much as a description of where the category still is. Good visibility is a real and necessary step — you can’t govern what you can’t see. The mistake isn’t buying a visibility tool. The mistake is believing it’s the whole answer, and stopping there.
If you’re serious about governing what your AI agents are allowed to do — not just monitoring what they’ve already done — the twelve questions above are the place to start. Ask them of everyone you’re evaluating.
At Nirmata, this is the specific gap we built AIControls to close: a single policy engine that sits inside the request path across LLM calls, MCP tool calls, and direct HTTP traffic — evaluating the actual arguments of every call and enforcing decisions before execution, not after, no matter which of those three paths an agent takes. If unified runtime enforcement is the piece your current stack is missing, we’d welcome the conversation.
