Skip to content

All posts

KubernetesAI

Kubernetes admission webhook failure policy: Fail or Ignore

Ritesh Patel

Kubernetes: Kubernetes admission webhook failure policy: Fail or Ignore

The failurePolicy field sets what the Kubernetes API server does when an admission webhook cannot answer. Fail rejects the request, and it is the default. Ignore lets the request continue unchecked. For a policy engine guarding agent pods, Fail keeps unchecked objects out during an outage, at the cost of blocking writes until the webhook recovers.

What the API server does when a webhook fails

An admission webhook can answer yes, answer no, or fail to answer. The failurePolicy setting only applies to the third case. The Kubernetes docs list the errors it covers: network errors, timeouts and connection failures, a non-2xx or malformed HTTP response, and a request the API server cannot serialize. For mutating webhooks, a patch it cannot decode also counts.

An explicit rejection is different. If the webhook is reached and returns allowed: false, the request is denied whatever the policy says. So Fail does not mean “reject anything the webhook dislikes.” It means “reject anything the webhook could not evaluate.” Keeping those two cases apart is the core of this topic.

Fail and Ignore compared

Situation failurePolicy: Fail failurePolicy: Ignore
Webhook times out, is unreachable, or returns a non-2xx or malformed response Request rejected Request continues, webhook skipped
Webhook is reached and returns allowed: false Request rejected Request rejected
Default for admission webhooks Yes No
What it protects Objects are never admitted without the check Writes keep working while the webhook is down
What it costs Writes in scope fail during an outage Objects are admitted without the check running

Use the table as a checklist. For each webhook, ask which of the two right-hand costs you can live with. Most of the decision comes down to whether a missed object is harmful or merely untidy.

Set the timeout before you choose the policy

Each webhook call has a timeout. It defaults to 10 seconds, and timeoutSeconds accepts values from 1 to 30. When the timeout expires, the API server treats the call as a failure, and failurePolicy decides the outcome. A slow webhook with Fail is therefore as disruptive as a dead one. The good-practices page asks for a small timeout because webhooks add to API request latency and should evaluate in milliseconds.

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: agent-pods-policy
webhooks:
  - name: agent-pods.example.com
    timeoutSeconds: 5
    failurePolicy: Fail
    namespaceSelector:
      matchExpressions:
        - key: kubernetes.io/metadata.name
          operator: NotIn
          values: ["kube-system", "my-namespace"]
    sideEffects: None
    admissionReviewVersions: ["v1"]
    clientConfig:
      service:
        namespace: my-namespace
        name: my-webhook
    rules:
      - operations: ["CREATE", "UPDATE"]
        apiGroups: [""]
        apiVersions: ["v1"]
        resources: ["pods"]

Fine print that changes the outcome

Match conditions have their own failure rule. A matchConditions expression that is false skips the webhook, whatever happens in the other expressions. If a match condition errors and none is false, Fail rejects the request and Ignore skips the webhook. A reader who assumes Ignore always means “admit” will miss this path.

Dependency loops are the second trap. The good-practices page describes a webhook that admits Pods only when they carry a label, where the webhook’s own Deployment does not set that label. When the node running the webhook fails, the Deployment cannot reschedule its pods, because the webhook rejects them for the missing label. Exclude the webhook’s own namespace with a namespaceSelector, and avoid policies that depend on the add-ons they validate. For the pod-level side of the same workloads, see Pod Security Admission levels for AI agent workloads.

Choosing Fail per webhook

  1. List what is registered: kubectl get validatingwebhookconfigurations and kubectl get mutatingwebhookconfigurations. Third-party installers often add webhooks you did not write.
  2. For each webhook, decide whether a missed object causes harm you cannot reverse. If so, use Fail. If the check is advisory, use Ignore.
  3. Scope it. Set a namespaceSelector to the namespaces that run agents, and avoid kube-system and the webhook’s own namespace.
  4. Set timeoutSeconds as low as the policy allows, inside the 1 to 30 second range.
  5. Roll it out in a test namespace first. The good-practices page gives this order for mutating webhooks: install with Ignore, watch the test namespace, then widen scope and switch to Fail. The page does not say the same order applies to validating webhooks, but it is a reasonable default for them too.
  6. Alert on rejections caused by the webhook failing. The API server metric apiserver_admission_webhook_rejection_count carries an error_type label, and calling_webhook_error marks rejections caused by an error calling the webhook. The docs mark the metric as alpha.

Enforcing this with Kyverno

Kyverno’s ValidatingPolicy docs cover the timeout side of this decision. The spec.webhookConfiguration.timeoutSeconds field sets how long the admission request waits for policy evaluation. The page gives the default as 10 seconds and the allowed range as 1 to 30. Kyverno reflects the value in the ValidatingWebhookConfiguration it generates.

The block below is a fragment that shows only the webhook settings. Merge it into a complete ValidatingPolicy with its own resource rules and validation expressions, and test that policy on a stated Kyverno version.

apiVersion: policies.kyverno.io/v1
kind: ValidatingPolicy
metadata:
  name: check-deployment-labels
spec:
  webhookConfiguration:
    timeoutSeconds: 15

The same page says that after the timeout, the request may fail or be ignored based on the failure policy. It does not state which failure policy a Kyverno policy gets, so read the failurePolicy on the generated ValidatingWebhookConfiguration in your cluster before you rely on either outcome. For the policy engine itself, see the Kyverno open source project. For the wider model of agent controls, start at governing AI agents, and compare the admission side with admission control vs runtime enforcement for AI agents.

Sources

Frequently asked questions

What is the default failurePolicy for a Kubernetes admission webhook?

Fail. The API server rejects the request when the webhook call errors, times out or returns a response it cannot use. Ignore is the other allowed value. You have to set it on each webhook configuration if you want the API server to continue without the webhook.

Does failurePolicy: Ignore let a request through even when the webhook says no?

No. If the webhook is reached and returns allowed: false, the request is denied whatever failurePolicy says. Ignore only applies when the webhook cannot give an answer, because of a network error, a timeout, a non-2xx response or a malformed response. The request then continues without the check.

How long does the API server wait for an admission webhook?

The default timeout is 10 seconds. You can set timeoutSeconds on the webhook from 1 to 30 seconds. If the timeout expires first, the call counts as a failure, so the failure policy decides whether the request is rejected or continues. Short timeouts keep a slow policy engine from holding up every write in its scope.

Should a policy engine that guards AI agent pods use Fail or Ignore?

Use Fail for a webhook that is the only check on a control you must keep, such as the label or security setting an agent pod needs. Pair it with a namespaceSelector so it covers only the namespaces that run agents, and with a short timeout. Use Ignore for convenience webhooks, where a missed check is an acceptable outage.

Ritesh Patel, Founder & VP Product, Nirmata

Ritesh Patel is a founder of Nirmata and leads its products. He writes about governing AI coding agents, MCP security and Kubernetes policy as code.

LinkedIn

Govern AI agents with AIControlsControl what AI agents do, and prove it.
Share

Stay up-to-date with the latest in AI, governance, and infrastructure

Subscribe to our free monthly newsletter to get industry news, DevOps tips, and articles right to your inbox on Kyverno, containerization, Kubernetes deployment & management and more.