How to detect shadow AI: unapproved model APIs and local LLMs

Shadow AI is any model API or local LLM that runs outside the tools your platform team approved. To detect it, check three places: the images of running pods, the model-server APIs that answer on your hosts, and the egress logs that show which model APIs people call. No single check is complete, so run all three.
Checked against the kubernetes.io docs (kubectl quick reference, Pods, and Container images) and the Ollama API reference, fetched 2026-11-14. The Kubernetes steps do not depend on a release. The Ollama reference does not pin a server release: its version example shows 0.5.1, so the Ollama steps describe the API as documented on the main branch of the ollama/ollama repository.
Where shadow AI shows up
Each surface gives different evidence, and each one misses a different case. Start with the cluster inventory, then probe, then read the logs.
| Surface | What to look for | What it misses |
|---|---|---|
| Pod images in the cluster | Model-server names in container and init container images | Renamed images, images referenced only by digest, and models pulled after the pod started |
| Model-server APIs on your hosts | An Ollama API answering on port 11434, with the models it lists | Servers on other ports, and hosts your probe cannot reach |
| Egress logs | Calls to model API hosts that are not on your approved list | Traffic that does not pass the logged path, including laptop traffic that skips it |
Pulls matter here. The Ollama API has a Pull a Model endpoint, so a model can arrive on a host after the image was approved. An inventory taken at deploy time will not see that, which is why Step 2 reads the live server.
Step 1: Inventory model-server images in the cluster
The How to find MCP servers in your codebase and clusters post inventories pod images for MCP servers. The steps below run the same kind of inventory for model servers.
- List the image of every container in every namespace, then filter for the model-server name.
kubectl get pods -A -o custom-columns='NAME:.metadata.name,IMAGE:.spec.containers[*].image' | grep -i ollama
- Run the same query against init containers. The
spec.containersquery does not read them. The Pods page listsspec.initContainers[*].imageas a separate field.
kubectl get pods -A -o custom-columns='NAME:.metadata.name,INIT_IMAGE:.spec.initContainers[*].image' | grep -i ollama
- Record the tag or digest for each hit. The Container images page says Kubernetes assumes the tag
latestwhen none is given, and that an image digest is a hash such assha256:.... The image name alone does not show which build runs, so compare each hit with your approved image list by digest where you can.
Step 2: Probe model servers on the hosts that can reach them
The Ollama API reference uses http://localhost:11434 in every example, so that is the port to check. The three endpoints below are all GET requests, and none of them changes the server.
curl http://localhost:11434/api/version
curl http://localhost:11434/api/tags
curl http://localhost:11434/api/ps
- Run
/api/versionfirst. A JSON reply with aversionfield confirms that an Ollama API is answering on that port. - Run
/api/tags. The reference describes it as the list of models available locally. Each entry gives a name, a size, a family, a parameter size and a quantization level. Record every name. - Run
/api/ps. The reference describes it as the list of models currently loaded into memory. An installed model that is idle shows up in tags but not here, so compare the two lists.
A pod is a group of containers that share network resources. The Pods page says so directly. A model server running as a sidecar therefore answers on localhost to the application container next to it. The inventory in Step 1 shows that sidecar as one more image in the same pod, not as a separate workload, so read both results before you assign an owner.
Step 3: Check egress for unapproved model APIs
Egress logs are the surface that can show calls to hosted model APIs, and neither Step 1 nor Step 2 reads that traffic. A model server can be idle while a workload calls a hosted API, so this step catches what the first two miss.
- Write down the model API hosts that your approved gateway and policies allow. Keep the list in version control so it can be reviewed.
- Pull DNS or proxy logs for the same time window as the probe. Filter for destinations outside the list.
- Group the hits by source. A pod that calls an unlisted host is one finding. A laptop that does the same is a separate finding, because the cluster inventory cannot see it.
- Compare the results with Steps 1 and 2. A workload that calls an unlisted host and also runs a model server is the strongest signal you will get from these checks.
Triage checklist for a hit
Use this list to decide what each hit needs. Record the answer next to the finding so the next review can see the reasoning.
- Is the model server in a namespace and image your team approved? If yes, record the image approval, then still run the other checks in this list. An approved image can pull an unapproved model and call an unlisted provider.
- Does
/api/tagslist a model that is not on your approved list? Flag it for the workload owner. - Can the server be reached from outside the workload that owns it? Escalate, because the server may be serving other callers.
- Does egress show calls to an unlisted model API from the same workload? Escalate, and check which credentials that workload uses.
Enforcing this with AIControls
Detection finds shadow AI after it runs. The lasting fix is to move approved developer traffic onto paths you can see and to give each user an identity. AIControls covers Claude Code via the gateway (CLI and VS Code), Claude Cowork and Claude on Bedrock, and it issues per-user tokens, so that usage maps to a user. The steps in this post do not depend on AIControls, and this post does not claim that AIControls detects unapproved endpoints or local models. For the wider governance picture, see governing AI coding agents and the AIControls platform page.
Sources
Frequently asked questions
Can I find local LLMs without installing software on every laptop?
Partly. The cluster inventory and the API probe cover workloads and hosts you run. A laptop running a model server is outside both. For laptops you need endpoint telemetry or a network log that shows the traffic. The egress step in this guide only sees traffic that passes a log you control, so treat a clean result as a statement about that path, not about every device.
Is an empty /api/ps a sign that no model is running?
It means no model is loaded into memory when you ask. The Ollama API reference describes /api/ps as the list of models currently loaded into memory, and /api/tags as the list of models available locally. A model can be installed and idle, so check both endpoints before you mark a host as clean.
Why not search only for the image name?
Kubernetes assumes the tag latest when an image has no tag, and an image can be referenced by a sha256 digest. A name search tells you nothing about which build runs, and it misses an image that was renamed. Record the tag or digest for each hit, and compare it with the digests on your approved image list.
Does the pod image query see init containers?
Not with the spec.containers query. Init containers have their own field, spec.initContainers, and an image listed there is not in spec.containers. Run a second query against the init field, which is why the guide runs both. An empty result from the first query does not clear a workload, so check both fields before you mark it as clean.
Is every shadow AI hit a policy violation?
No. A hit is a finding to classify. Some are tests that a team forgot to remove, and some are approved work that never reached the list. The checks show what a server exposes and which host it calls. They do not show who approved it, so the owner of the workload has to answer that.