In this post I will show you how to audit your clusters for compliance with the latest Kubernetes Pod Security Standards without installing anything in the cluster.
Pods are the basic unit of execution in Kubernetes, and pod security is necessary for all clusters. Without pod security checks enabled, any user with permissions to run a pod can elevate privileges. Attackers can exploit the lack of pod security to execute a container escape. All clusters, including dev-test and staging clusters which are a common entry-point for attackers, should implement pod security.
The Kubernetes project publishes the Pod Security Standards which contains security controls, organized into three profile levels, that should be enforced.
Kubernetes v1.25 offers an in-tree admission controller for the pod security standards, which offers namespace-level validation and enforcement and needs to be configured at the API server. In most cases, more granular controls will be needed. This blog post provides a good analysis of pod security admission.
To check for compliance with the Kubernetes Pod Security Standards, we will run the Kyverno CLI from outside the cluster and execute policies for each of the controls defined in the pod security standards. To perform the audit you will need access to the cluster via
kubectl, but do not need to install anything in the cluster.
Step 1: Install krew and kustomize if needed
Krew is a package manager for
kubectl the Kubernetes CLI. You can install it using the instructions at:
Kustomize is a
kubectl subcommand that simplifies configuration management. Since the version distributed with
kubectl tends to be old, you can install the latest from:
Step 2: Install the Kyverno kubectl plugin
Next, install the
kyverno kubectl plugin:
kubectl krew install kyverno
The output should look like this:
Updated the local copy of plugin index. Installing plugin: kyverno Installed plugin: kyverno \ | Use this plugin: | kubectl kyverno | Documentation: | https://github.com/kyverno/kyverno | Caveats: | \ | | The plugin requires access to create Policy and CustomResources | / / WARNING: You installed plugin "kyverno" from the krew-index plugin repository. These plugins are not audited for security by the Krew maintainers. Run them at your own risk.
Step 3: Scan your cluster
kyverno command line as follows:
kustomize build https://github.com/kyverno/policies/pod-security | kubectl kyverno apply --cluster -
The above command runs against the entire cluster. You can optionally use the
--namespace option to scan a single namespace.
Here is the output from my cluster’s default namespace where I ran a
❯ kubectl run busybox --image busybox pod/busybox created
❯ kustomize build https://github.com/kyverno/policies/pod-security | kubectl kyverno apply --cluster --namespace default -
Applying 17 policies to 1 resource... policy disallow-capabilities-strict -> resource default/Pod/busybox failed: 1. require-drop-all: validation failure: Containers must drop `ALL` capabilities. policy disallow-privilege-escalation -> resource default/Pod/busybox failed: 1. privilege-escalation: validation error: Privilege escalation is disallowed. The fields spec.containers[*].securityContext.allowPrivilegeEscalation, spec.initContainers[*].securityContext.allowPrivilegeEscalation, and spec.ephemeralContainers[*].securityContext.allowPrivilegeEscalation must be set to `false`. Rule privilege-escalation failed at path /spec/containers/0/securityContext/ policy require-run-as-nonroot -> resource default/Pod/busybox failed: 1. run-as-non-root: validation error: Running as root is not allowed. Either the field spec.securityContext.runAsNonRoot must be set to `true`, or the fields spec.containers[*].securityContext.runAsNonRoot, spec.initContainers[*].securityContext.runAsNonRoot, and spec.ephemeralContainers[*].securityContext.runAsNonRoot must be set to `true`. Rule run-as-non-root failed at path /spec/securityContext/runAsNonRoot/. Rule run-as-non-root failed at path /spec/containers/0/securityContext/. policy restrict-seccomp-strict -> resource default/Pod/busybox failed: 1. check-seccomp-strict: validation error: Use of custom Seccomp profiles is disallowed. The fields spec.securityContext.seccompProfile.type, spec.containers[*].securityContext.seccompProfile.type, spec.initContainers[*].securityContext.seccompProfile.type, and spec.ephemeralContainers[*].securityContext.seccompProfile.type must be set to `RuntimeDefault` or `Localhost`. Rule check-seccomp-strict failed at path /spec/securityContext/seccompProfile/. Rule check-seccomp-strict failed at path /spec/containers/0/securityContext/. pass: 15, fail: 4, warn: 0, error: 0, skip: 38
The output above shows that the busybox pod violates four controls in the pod security standards.
Kyverno is a powerful and simple tool for Kubernetes security and automation. It typically runs as an admission controller, in the Kubernetes control-plane.
The Kyverno CLI can execute Kyverno policies against a set of files containing Kubernetes resource YAML declarations, or can execute policies against a cluster. Here, we used the Kyverno CLI to execute policies that implement the Pod Security Standards against a cluster.
As a next step, you can install Kyverno in your cluster. Or, try the Nirmata Kubernetes Policy Manager which measures compliance to pod security standards, NIST 800-53, the NSA Kubernetes Guidelines, CIS Kubernetes Benchmarks, and more.
A free trial of the Nirmata Policy Manager is available at: https://try.nirmata.io/.