In this post I will show you how to audit your clusters for compliance with the latest Kubernetes Pod Security Standards without installing anything in the cluster.

Pods are the basic unit of execution in Kubernetes, and pod security is necessary for all clusters. Without pod security checks enabled, any user with permissions to run a pod can elevate privileges. Attackers can exploit the lack of pod security to execute a container escape. All clusters, including dev-test and staging clusters which are a common entry-point for attackers, should implement Kubernetes pod security.

The Kubernetes project publishes the Pod Security Standards  which contains security controls, organized into three profile levels, that should be enforced.

Kubernetes v1.25 offers an in-tree admission controller for the pod security standards, which offers namespace-level validation and enforcement and needs to be configured at the API server.  In most cases, more granular controls will be needed. This blog post provides a good analysis of Kubernetes pod security admission.

To check for compliance with the Kubernetes Pod Security Standards, we will run the Kyverno CLI from outside the cluster and execute policies for each of the controls defined in the pod security standards. To perform the audit you will need access to the cluster via kubectl, but do not need to install anything in the cluster.

Step 1: Install krew and kustomize if needed

Krew is a package manager for kubectl the Kubernetes CLI. You can install it using the instructions at:

https://krew.sigs.k8s.io/docs/user-guide/setup/install/

Kustomize is a kubectl subcommand that simplifies configuration management. Since the version distributed with kubectl tends to be old, you can install the latest from:

https://kubectl.docs.kubernetes.io/installation/kustomize/binaries/

Step 2: Install the Kyverno kubectl plugin

Next, install the kyverno kubectl plugin:

kubectl krew install kyverno

The output should look like this:

Updated the local copy of plugin index.
Installing plugin: kyverno
Installed plugin: kyverno
\
| Use this plugin:
| kubectl kyverno
| Documentation:
| https://github.com/kyverno/kyverno
| Caveats:
| \
| | The plugin requires access to create Policy and CustomResources
| /
/
WARNING: You installed plugin "kyverno" from the krew-index plugin repository.
These plugins are not audited for security by the Krew maintainers.
Run them at your own risk.

Step 3: Scan your cluster

Run the kyverno command line as follows:

kustomize build https://github.com/kyverno/policies/pod-security | kubectl kyverno apply --cluster -

The above command runs against the entire cluster. You can optionally use the --namespace option to scan a single namespace.

Here is the output from my cluster’s default namespace where I ran a busybox image:

❯ kubectl run busybox --image busybox
pod/busybox created

❯ kustomize build https://github.com/kyverno/policies/pod-security | kubectl kyverno apply --cluster --namespace default -

Applying 17 policies to 1 resource...

policy disallow-capabilities-strict -> resource default/Pod/busybox failed:
1. require-drop-all: validation failure: Containers must drop `ALL` capabilities.

policy disallow-privilege-escalation -> resource default/Pod/busybox failed:
1. privilege-escalation: validation error: Privilege escalation is disallowed. The fields spec.containers[*].securityContext.allowPrivilegeEscalation, spec.initContainers[*].securityContext.allowPrivilegeEscalation, and spec.ephemeralContainers[*].securityContext.allowPrivilegeEscalation must be set to `false`. Rule privilege-escalation failed at path /spec/containers/0/securityContext/

policy require-run-as-nonroot -> resource default/Pod/busybox failed:
1. run-as-non-root: validation error: Running as root is not allowed. Either the field spec.securityContext.runAsNonRoot must be set to `true`, or the fields spec.containers[*].securityContext.runAsNonRoot, spec.initContainers[*].securityContext.runAsNonRoot, and spec.ephemeralContainers[*].securityContext.runAsNonRoot must be set to `true`. Rule run-as-non-root[0] failed at path /spec/securityContext/runAsNonRoot/. Rule run-as-non-root[1] failed at path /spec/containers/0/securityContext/.

policy restrict-seccomp-strict -> resource default/Pod/busybox failed:
1. check-seccomp-strict: validation error: Use of custom Seccomp profiles is disallowed. The fields spec.securityContext.seccompProfile.type, spec.containers[*].securityContext.seccompProfile.type, spec.initContainers[*].securityContext.seccompProfile.type, and spec.ephemeralContainers[*].securityContext.seccompProfile.type must be set to `RuntimeDefault` or `Localhost`. Rule check-seccomp-strict[0] failed at path /spec/securityContext/seccompProfile/. Rule check-seccomp-strict[1] failed at path /spec/containers/0/securityContext/.

pass: 15, fail: 4, warn: 0, error: 0, skip: 38

Conclusion

Kyverno is a powerful and simple tool for Kubernetes security and automation. It typically runs as an admission controller, in the Kubernetes control-plane.

The Kyverno CLI can execute Kyverno policies against a set of files containing Kubernetes resource YAML declarations, or can execute policies against a cluster. Here, we used the Kyverno CLI to execute policies that implement the Pod Security Standards against a cluster.

As a next step, you can install Kyverno in your cluster. Or, try the Nirmata Kubernetes Policy Manager, which measures compliance to Kubernetes pod security standards, NIST 800-53, the NSA Kubernetes Guidelines, CIS Kubernetes Benchmarks, and more.

A free trial of the Nirmata Policy Manager is available at: https://try.nirmata.io/. We would also be delighted to answer any specific questions you have regarding Kyverno, pod security or our Policy Manager for DevOps needs. Please reach us here to start a conversation. Thanks for reading.