Securing software supply chains remains a top priority for DevOps teams. In a prior post, we discussed the A MAP framework for Kubernetes supply chain security.  In this post, we will discuss how supply chain security can be implemented using Nirmata Policy Manager and Venafi CodeSign Protect. 

With supply chain attacks on the rise, securing the software supply chain has become a requirement for DevOps teams building cloud-native applications on Kubernetes. Signing the container image is typically the very first step in software supply chain security. The modern application development and deployment process is highly automated with platforms like Kubernetes & Github at the heart of continuous integration (CI) and continuous delivery (CD) practices. Any additional steps required for signing and verifying container images need to be automated as well so that software development agility can be maintained. 

About Venafi CodeSign Protect (or Venafi TPP platform)

Venafi CodeSign Protect helps you manage all code signing private keys and automatically enforce code signing processes in a fast, secure, and consistent manner for your development teams. It reduces the risk of your code signing credentials getting into the hands of cybercriminals. CodeSign Protect automates the lifecycle of signing keys and certificates from issuance to revocation. In addition, the Security team can define and enforce code signing best practices across all development teams for an enterprise. CodeSign Protect also easily integrates into existing build pipelines allowing developers to use native signing tools compliant with DevStack.

About the Nirmata Policy Manager

Nirmata Policy Manager lets DevSecOps teams ensure the security, compliance, and operational readiness of their Kubernetes Workloads and Clusters by; automating the creation, deployment, and lifecycle management of policy-based Intelligent Guardrails, delivering policy insights, alerts, and reports, and facilitating team collaboration through integration with DevSecOps process, tools, and workflows. You can create and deploy policies to verify image signatures ensuring that only verified images are deployed to your clusters. 

Putting it all together

Here are the steps to set up supply chain security in your CI/CD pipeline.

1. Create the certificate
2. Sign the container image
3. Create the policy

Create the certificate

Configure the code signing certificate for container image signing in Venafi CodeSign Protect.

1. Login to the Venafi CodeSign Protect UI (e.g. https://someorg-tpp.se.venafi.com/codesign-protect/) as Team Owner
2. Navigate to the “Projects” page and create a new project for your development team.
3. Assign users with their designated roles in accordance with the security team’s policies. For example, (1) Key User is a developer or service account that will perform the signing operation. 
4. Click on “Add Environment->Certificate and Key”
5. Select Creation Type as “Create New” or “Import PKCS#12/PFX”, and provide the information to create a code signing certificate. 

Add Certificate

Note: The above steps are for Venafi TPP platform version 22.1. Details might vary based on the platform version.

Sign the container image

Next, sign the container image using the Venafi Codesign Protect client (https://www.venafi.com/platform/codesign-protect).

Install Venafi Codesign Protect Client and Cosign

1. Install the Venafi Codesign Protect Client for your platform. The client is available for Linux, Windows and MacOS platforms. Install instructions are provided in the Venafi Codesign Protect Help Documentation (Typically at: Venafi Documentation > CodeSign Protect > Installing CodeSign Protect > Install CodeSign Protect Clients on signing workstations) 
2. Download the cosign v1.8.0 pkcs11 binary from the github repo: https://github.com/sigstore/cosign/releases/download/v1.8.0/cosign-linux-pivkey-pkcs11key-amd64

Sign image with Cosign

First, run the following command to get the OAUTHtoken.

/opt/venafi/codesign/bin/pkcs11config getgrant --force --username <key-user name> --password <key-user password> --authurl https://myorg-tpp.se.venafi.com/vedauth --hsmurl https://myorg-tpp.se.venafi.com/vedhsm

The list of code signing certificates corresponding to the OAUTH token can be seen by running the following command. 

pkcs11config listobjects -tab 

Select the certificate & sign the container image using cosign and PKCS11 module provided by Venafi CodeSign Protect client. 

cosign sign --key "pkcs11:slot-id=0;object=prod-1?module-path=/opt/venafi/codesign/lib/venafipkcs11.so&pin-value=<PIN>" ghcr.io/kyverno/test-verify-image:signed

Create the policy

Now, you need to create a policy that can verify the image signature whenever a pod using that image is deployed in a k8s cluster.

Install Kyverno and Venafi key-fetch controller

If your cluster does not contain Kyverno 1.7. You need to install it: 

kubectl create -f https://raw.githubusercontent.com/kyverno/kyverno/v1.7.0-rc3/config/release/install.yaml

(You might need to uninstall a previous version of Kyverno if it exists)

Install the Venafi key controller on the cluster by running: 

kubectl create -f https://raw.githubusercontent.com/nirmata-add-ons/imagekey-controller/main/deploy.yaml

This creates an image key-fetch controller in the “nirmata” namespace.

Configure Venafi CodeSign Protect in NPM

Configure the access information for Venafi CodeSign Protect in the Nirmata Policy Manager. This can be done in the Integrations tab. You will need the following information to configure the access:

• Name – a unique identifier for this account
• Auth URL – The authentication URL for the Venafi Trust Protection Platform (TPP).
• HSM URL – The HSM URL for the Venafi Trust Protection Platform (TPP).
• Username – The username to access the Venafi Trust Protection Platform (TPP).
• Password – Password for this account
• Additional Trusted Certificate – This is optional and only needed if using a self-signed certificate or a private CA certificate for the Venafi Trust Protection Platform (Usually in test environments).

Add Account

Create Image Verification Policy

To create an image verification policy for Venafi CodeSign Protect, go to your Policy Group and select “Add Policy”. This will launch the policy editor. In the editor, select “Venafi Key” in the Examples dropdown. Next, specify the required information in the dialog:

• Policy name – enter a unique name for your policy
• Images – specify the images to which this policy should be applied
• Venafi Account and Label – select the account and specify the label for the key. This label will be used to fetch the key from Venafi CodeSign Protect. Multiple allowed key labels can be associated with the policy allowing users to download containers signed by multiple sources.

Once the information is added, you will see the policy YAML in the editor. You can continue making any other changes to the policy in the editor.

Next, deploy the policy to your cluster. You can verify that the policy has been deployed running this command:

kubectl get cpol <policy-name>

Check the policy to make sure it is in a “READY” state

Verify the policy

Once the policy has been deployed, we are ready to test the end-to-end flow.

1. Deploy a pod that uses the unsigned image. The deployment for this pod should be blocked due to signature mismatch. In Nirmata Policy Manager, you will see a policy violation with the details.

kubectl run unsigned --image=ghcr.io/kyverno/test-verify-image:unsigned

1. Deploy a pod that uses the signed image & is compliant with the image verification policy. The deployment for this pod should be successful.

kubectl run signed --image=ghcr.io/kyverno/test-verify-image:signed

As you can see, any image not compliant with the image verification policy will be blocked. The policy can be scoped at the cluster or namespace level. 

Summary

As supply chain attacks are becoming increasingly common, it has become absolutely necessary to secure various phases in your CI/CD pipeline. Signing your container images prior to deploying them to your Kubernetes is the first step. Now, you can easily automate this process using Venafi CodeSign Protect and Nirmata Policy Manager.

I hope this blog post helps you understand how to secure your container images using Nirmata Policy Manager and Venafi CodeSign Protect. Please reach out to us if you have any questions or need any help with your software supply chain security.