OpenShell credentials and inference routing explained

OpenShell never gives an agent the real API key. Inside the sandbox, each credential is an opaque placeholder, and the supervisor swaps in the real value only when network policy allows the request and the provider profile authorizes that endpoint. For model access, the workload calls the provider’s native API and picks its own model.

This post is checked against the OpenShell v0.1.3 docs, fetched on 2026-11-10. It covers how a credential reaches a request, how inference access works now, and how to debug a denied call. For the sandbox setup itself, read our OpenShell on Kubernetes guide. We link to it rather than repeat it. For how OpenShell compares with other sandbox runtimes, see OpenShell vs Agent Substrate vs Kubernetes Agent Sandbox. For the wider pattern, see how to sandbox AI agents on Kubernetes.

Why the agent never sees the key

If an agent reads its API key from an environment variable, any code the agent runs can read that key. OpenShell changes what the agent’s environment holds. At startup, it replaces each credential with an opaque placeholder token. The agent can put the placeholder in a request, but the placeholder is not the secret.

The real value stays on the trusted side of the boundary. The supervisor checks each request against policy and adds credentials only where policy allows. The sandbox shares the boundary with the untrusted agent, so it holds no provider credentials and no gateway signing key. The architecture page says the sandbox holds nothing worth stealing.

That split is what makes a placeholder safe to hand to untrusted code. An agent that sends one to the wrong host gets a 403, not the key.

How a placeholder becomes a credential

A request that carries a placeholder passes five steps before it leaves the sandbox. Each step can fail closed.

1. At startup, OpenShell sets the placeholder in the agent’s environment. A process that started before you attached the provider keeps its old environment, so start a new process.
2. The agent sends an HTTP request that contains the placeholder.
3. The supervisor checks network policy for the calling program and the destination. It identifies the program from trusted process data, not from a path the agent supplies.
4. The credential binding checks the request’s host, port and path against the endpoints in the provider profile.
5. If both checks pass, the proxy replaces the placeholder and forwards the request. If policy allows the request but the binding does not, the proxy returns HTTP 403 with the reason credential_endpoint_mismatch.

Unknown, malformed or expired placeholders also fail closed. They are never forwarded upstream. The proxy substitutes placeholders only in the locations below. It does not rewrite cookies, response bodies, unsupported request bodies or binary WebSocket frames.

Location	How the agent uses it	Example
Header value	Reads the variable and puts it in a header	Bearer token in the Authorization header
Basic auth	Base64-encodes user:placeholder; the proxy decodes, resolves and re-encodes	Authorization: Basic header
Query parameter	Puts the placeholder in a URL query value	An API key in a key parameter
URL path segment	Builds the URL with the placeholder in the path	A bot token inside the path
Request body (opt-in)	The REST endpoint sets request_body_credential_rewrite: true	A JSON api_key field
AWS SigV4	The endpoint sets credential_signing: sigv4	The proxy signs with endpoint-bound AWS credentials

Create a provider and attach it to a sandbox

A provider is an instance of an imported profile. The profile names the credential variable, the endpoints and the binaries allowed to call them. Start from the NVIDIA example profile in the providers directory. Change id to nvidia-native, give it a distinct display_name, and set binaries to the programs that may call the API. For a Python workload, the binaries can look like this:

id: nvidia-native
display_name: NVIDIA Native API
binaries:
  - /usr/bin/python3.*
  - /usr/local/bin/python3.*
  - /sandbox/.venv/**

Then lint and import the profile, create the provider and start a sandbox with it:

openshell profile lint -f nvidia-native.yaml
openshell profile import -f nvidia-native.yaml
openshell provider create --name nvidia-prod --type nvidia-native --from-existing
openshell sandbox create --name inference-demo --from registry.example.com/team/python-agent:1.0 --provider nvidia-prod -- python app.py

To add the provider to a running sandbox, attach it with --wait. That flag confirms the sandbox has applied the change. Then start a new process, because an existing process does not gain new environment variables.

openshell sandbox provider attach inference-demo nvidia-prod --wait
openshell sandbox provider list inference-demo
openshell sandbox exec --name inference-demo -- python app.py
openshell policy get inference-demo --full

openshell policy get --full shows the effective policy, including the entries derived from the provider. Check it before you assume a call will pass.

Inference without a shared route

Earlier releases configured one route per workspace. That route named a single provider and model, every sandbox used it, and clients called a managed virtual endpoint. The v0.1.3 inference page describes a different model. Provider attachment grants access to selected sandboxes, and the workload calls the provider’s native endpoint.

The v0.1.3 provider overview lists these example profiles and their native endpoints:

Provider	Example profile	Native base URL	Credential variable
OpenAI	openai	https://api.openai.com/v1	OPENAI_API_KEY
Anthropic	anthropic	https://api.anthropic.com	ANTHROPIC_API_KEY
NVIDIA API Catalog	nvidia	https://integrate.api.nvidia.com/v1	NVIDIA_API_KEY
DeepInfra	deepinfra	https://api.deepinfra.com/v1/openai	DEEPINFRA_API_KEY
Google Vertex AI	google-vertex-ai	Depends on region and model	GOOGLE_VERTEX_AI_TOKEN or GOOGLE_VERTEX_AI_SERVICE_ACCOUNT_TOKEN

The client sets the model and the timeout. Nothing in the attachment does. The docs’ own example looks like this:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://integrate.api.nvidia.com/v1",
    api_key=os.environ["NVIDIA_API_KEY"],
    timeout=300,
)

response = client.responses.create(
    model="nvidia/nemotron-3-super-120b-a12b",
    input="Hello!",
)
print(response.output_text)

The OpenShell environment supplies NVIDIA_API_KEY as a placeholder. The client code does not need to know that. This design gives the provider profile control over the allowed API surface. The workload keeps control of headers, model choice, request shape, streaming and timeouts.

Point a sandbox at a self-hosted endpoint

The openai profile binds its credential to the public OpenAI endpoint. If you point OPENAI_BASE_URL at another host, OpenShell treats the provider as endpointless and does not send the credential to that host. To reach a different host, import a profile that names it, along with the programs allowed to call it.

The docs’ example is a credentialless Ollama server on the gateway host:

id: ollama-openai
display_name: Ollama
description: Host-local Ollama OpenAI-compatible API
category: inference
inference_capable: true
credentials: []
endpoints:
  - host: host.openshell.internal
    port: 11434
    protocol: rest
    access: read-write
    enforcement: enforce
binaries:
  - /usr/bin/curl
  - /usr/local/bin/curl
  - /usr/bin/python3
  - /usr/local/bin/python
  - /sandbox/.uv/python/**
  - /sandbox/.venv/**

openshell profile lint -f ollama-openai.yaml
openshell profile import -f ollama-openai.yaml
openshell provider create --name ollama --type ollama-openai

openshell sandbox create \
  --name ollama-client \
  --from registry.example.com/team/python-agent:1.0 \
  --provider ollama \
  --env OPENAI_BASE_URL=http://host.openshell.internal:11434/v1 \
  -- python app.py

A host-local service needs host.openshell.internal or a reachable service hostname. The docs say it does not work through 127.0.0.1 or localhost.

The Python client shown earlier targets the NVIDIA profile: it hardcodes that base URL and reads NVIDIA_API_KEY. The Ollama sandbox needs a client that reads OPENAI_BASE_URL, which the command above sets, and the workload image must contain that client. The docs do not show one, so write it to match your app, and choose a model your Ollama server has installed.

Move off a managed inference route

Upgrading removes the stored route records. It keeps provider records, provider refresh configuration and existing attachments. The old route cannot be converted automatically, because it applied one provider and model to every sandbox, and OpenShell cannot tell which sandboxes should get that access. The 0.1.0 upgrade notes require one release across every component, no mixed 0.0.x and 0.1.x peers, and removal of 0.0.x sandboxes before the upgrade. Split the move around the upgrade.

Before the upgrade, on the 0.0.x release

1. Record the old provider, model and timeout with the previous release’s openshell inference get.
2. Find the sandboxes that actually call the model. Do not assume it is all of them.
3. Export the provider’s profile before upgrading.
4. Remove the 0.0.x sandboxes. You recreate them after the upgrade.

openshell provider profile export <id> -o yaml --global > <id>.yaml

After the upgrade, on 0.1.3 everywhere

5. Edit the exported profile’s id, endpoints and binaries, then lint and import it under a new ID.
6. Create a replacement provider from the original credential source and attach it to each sandbox that calls the model.
7. Recreate each sandbox, reapply its attachments, and move each client to the native endpoint, the real model identifier and its own timeout.

# Edit id, display_name, endpoints, and binaries in nvidia-native.yaml.
openshell profile lint -f nvidia-native.yaml
openshell profile import -f nvidia-native.yaml
openshell provider create --name nvidia-native-prod --type nvidia-native --from-existing
openshell sandbox provider attach inference-demo nvidia-native-prod --wait

Code that still calls the retired managed virtual endpoint fails DNS resolution, because OpenShell no longer resolves or trusts that host. That failure is the signal to find the client you missed.

Where provider credentials are stored

By default, the gateway stores submitted credentials as AES-256-GCM encrypted envelopes in its database, outside the provider and refresh records. When no external credential driver is enabled, the Helm chart creates a retained Kubernetes Secret for the default key-encryption key and injects it into each gateway pod. A multi-replica Kubernetes gateway can use a shared database with that default store, or a shared backend such as kubernetes-secrets or vault. An empty credential_drivers list is invalid.

To keep values in Kubernetes Secrets, set server.credentialDrivers.kubernetesSecrets.enabled=true and a dedicated namespace with server.credentialDrivers.kubernetesSecrets.namespace. The gateway then gets Secret permissions through a Role in that namespace only. openshell provider get and provider list show key names and never values.

Changing the driver later means deleting or reconfiguring the affected providers while the original driver still works. Then select the new driver and create the credentials again.

Troubleshoot a denied request

When a call fails, check these in order:

1. Confirm the provider is attached with openshell sandbox provider list.
2. Confirm the calling binary and the native endpoint appear in openshell policy get with --full.
3. If you attached the provider after a process started, launch a new process.
4. If the sandbox uses a gateway global policy override, add the native endpoint to that override. A global override suppresses the policy layers that providers derive.
5. Search the logs for credential_endpoint_mismatch. That error means policy admitted the request, but the provider profile does not authorize the credential at that endpoint.

A missing provider profile also fails at sandbox creation, and the error names the profile and the import command. Treat that as a setup error, not a network one.

Enforcing this with AIControls

OpenShell’s injection protects credentials inside each sandbox. For coding agents that run on developer machines, AIControls supports per-user tokens and per-user cost and budget limits. Those controls sit beside sandbox injection and do not replace it. Before you decide who needs a per-user token, list the credentials each sandbox holds, and check which endpoints each one can reach. For admission-time controls on the Pods around your agents, see Kyverno policies for AI agents. Start with the AIControls platform or the AI agent governance hub.

Sources

• OpenShell providers overview: placeholders, credential binding, the 403 reason, example inference profiles.
• OpenShell inference: native calls, attach and detach, custom endpoints, managed-route migration.
• OpenShell provider profiles: credential storage, the encrypted default store, driver rules.
• OpenShell architecture: the supervisor and sandbox boundary, what the sandbox holds, fail-closed behaviour.
• OpenShell Kubernetes setup: Kubernetes Secrets credential driver and its Helm values.
• OpenShell extensibility overview: credential drivers as the link to the deployment’s secret store.

Frequently asked questions

Does an OpenShell agent ever see the real API key?

No. At startup, OpenShell puts an opaque placeholder for each credential into the agent's environment. The real value stays with the supervisor, which substitutes it into a request only after network policy and the provider profile's endpoint binding both pass. The sandbox also holds no provider credentials and no gateway signing key, so the agent's side of the boundary has nothing to read.

Does OpenShell choose which model my agent uses?

No. Attaching a provider grants a sandbox access and a credential placeholder. It does not select or rewrite a model. The workload calls the provider's native endpoint, sends the real model identifier and sets its own request timeout. Code that still calls the retired managed virtual endpoint fails DNS resolution, so migrate it to the native endpoint.

Why does a request return 403 with credential_endpoint_mismatch?

The network policy allowed the request, but the provider profile does not bind that credential to the host, port and path being called. Either send the request to an endpoint the profile names, or import a custom profile for the host. Do not reuse the openai or anthropic example profile for another host, because those bind credentials to the public vendor endpoints.

Can OpenShell keep provider credentials in Kubernetes Secrets?

Yes. Enable the Kubernetes Secrets credential driver in the Helm chart and give it a dedicated namespace. The gateway then gets Secret permissions only in that namespace. By default, the gateway stores credentials as AES-256-GCM encrypted envelopes in its database. Choose the driver before you create providers, because changing it means reconfiguring them.