Kubernetes FinOps Policies with Kyverno

Kubernetes FinOps Policies with Kyverno

andre taissin 5OUMf1Mr5pU unsplash

FinOps, short for Financial Operations, is the approach that drives collaboration between finance, technology, and business teams to maximize the value realization from cloud-native solutions using data-driven decision-making across the functions.  In recent times, Kubernetes has become the de-facto application orchestration and as many as 80% of enterprises have adopted it in production. While Kubernetes has delivered on its promise of cloud-native application orchestration, its dynamic nature, and the layer of abstraction create challenges in gaining visibility and optimizing resources. 

Today, there are many solutions that exist for gaining visibility into Kubernetes cost, but the decision-making on the data and action is still handled out of band, which is not well suited to implementing cost management in real time. 

This is where Kyverno comes in. Kyverno is a Kubernetes-native policy engine that enables the creation and enforcement of policies for Kubernetes resources. Kyverno policies define rules that govern the behavior of Kubernetes resources, including pods, deployments, and services. Kubernetes provides an admission control capability that allows for many decisions to be implemented. These policies can be used to enforce compliance, security, and cost optimization rules for Kubernetes resources. Kyverno implements audit and enforcement decisions as policies using the admission control capabilities Kubernetes provides.   

Kyverno’s capabilities are well suited to implement FinOps policies for Kubernetes as they can provide validation, enforcement, and triggered actions to implement decisions based on specific thresholds and events. Kubernetes FinOps policies can help organizations ensure compliance with internal policies and regulatory requirements. By implementing policies for image management, access control, and other areas, organizations can ensure that their Kubernetes environment is secure and compliant.

  1. Cost allocation: Cost allocation is required by finance to manage budgets for different departments. Labels and annotations can be used to assign the correct department labels to the resources so that when the cost is calculated for the resources, it can be grouped using these labels.  Labels can be used to provide additional information about a resource, such as its owner, project, or cost center. The Kyverno policies can be used not just to validate if appropriate labels are configured but also to attach labels in case labels are missing for certain resources. This ensures that both validation and implementation of resources are achieved via a set of Kyverno policies. Here is a Kyverno policy that performs a validation to check cost center labels on the resources.
  2. Quota Management for teams: One of the basic requirements for resource management is to ensure that all pods use requests and limits for CPU and memory and every namespace has a quota assigned to ensure that misconfigured resources don’t bring down the entire cluster. The quotas can also be used to allocate resources based on the application budget.  Here are some sample Kyverno policies that can validate namespace quotas exist and, if not,  can create ns quota, request, and limit based on a condition. There are also policies that can validate all the containers have limits specified for memory and CPU.
  3. Implement scaling policies: One problem often reported by platform engineering teams is the misconfigured scaling of applications resulting in unexpected cost and performance issues.  Pod controllers such as Deployments, Statefulsets, etc. that implement replicas and permit the scale action use a `/scale` subresource to control this behavior. In addition to checks for creations of such controllers that their replica is in a certain shape, the scale operation and subresource need to be accounted for as well. To ensure that a misconfiguration does not result in unexpected scaling of resources, a Kyverno policy, which is a set of rules that can be cherry-picked based on the use case, can act as a guardrail.
  4. Validate and configure auto-scaling: Horizontal Podlicy Autoscaling (HPA) and Vertical Pod Autoscaling (VPA) are essential components to scale Kubernetes workloads based on performance requirements which helps to optimize resource utilization, improve application availability and scalability, and reduce operational costs. Kyverno policies can be used to both validate auto-scaling but can also be used to configure auto-scaling in case not configured. Here are some sample Kubernetes Autoscaler Kyverno policies that ​​ensures the application can handle sudden spikes in traffic without downtime, and can also scale down when the demand decreases, which saves resources and reduces costs.
  5. Clean up unused resources:  By any conservative estimate, unused resources are contributing to 30-50% of cost overhead. In Kubernetes clusters, removing unused resources is especially hard due to a lack of understanding of application ownership, its criticality, and an easy way to clean up the resources granularly. Kyverno provides a resource cleanup policy that can be used to clean up resources based on cost, state, consumption, time lease, staleness, etc. Here is a sample Kyverno cleanup policy that cleans up resources based on certain labels and replica counts which organizations can use to keep their systems clean, well-organized, and efficient. Apart from helping manage costs, cleanup policies also ensure security by removing unnecessary data that could be used by attackers to gain unauthorized access to systems or sensitive information.
  6. Cleanup of orphaned pods: Pods not created by workload controllers such as Deployments have no self-healing or scaling abilities and are unsuitable for production and can easily get accumulated over time, resulting in additional costs. Identifying the orphaned pods is not straightforward but a Kyverno policy can be used to identify such pods and delete them.    
  7. Scale down resources in crash loop: If a Deployment’s Pods are seen crashing multiple times it usually indicates there is an issue that must be manually resolved. Scaling down such resources in Kubernetes can help you optimize costs, improve performance, efficiently utilize resources, and avoid over-provisioning. By regularly monitoring and scaling down resources that are not being fully utilized, you can ensure that your application is running efficiently and cost effectively. This sample Kyverno policy watches existing Pods and if any are observed to have restarted more than once, indicating a potential crash loop, Kyverno scales its parent deployment to zero and writes an annotation signaling to an SRE team that troubleshooting is needed.
  8. Limit image sizes: Containers running large images take longer to pull and consume more storage and bandwidth, increasing cloud costs. Restricting large images reduces unnecessary network and storage costs. A user may either inadvertently or purposefully name an image that is unusually large to disrupt operations. This sample Kyverno policy checks the size of every container image and blocks if it is over 2 GB.


Kyverno provides a flexible and powerful way to implement Kubernetes FinOps policies, allowing organizations to not just monitor but implement cost management measures for Kubernetes. By using Kyverno to implement Kubernetes FinOps policies, organizations can optimize their costs, improve their resource utilization, ensure compliance, increase agility, and reduce operational overhead related to their Kubernetes environment.

FinOps Policies Management with Nirmata

Nirmata offers Kubernetes governance with Policy Management as the key pillar. Our cloud-native policy management solution, powered by Kyverno, facilitates the autonomy, agility, and alignment necessary for DevSecOps teams, by automating the creation, deployment, and lifecycle management of policy-based intelligent guardrails. Nirmata delivers policy insights, reports, tamper detection, alerts, and collaboration by integrating with external tools, processes, and workflows. Nirmata offers an Enterprise distribution of Kyverno and SaaS based Nirmata Policy Manager. A free trial for both products is available. Let us know what you think about Nirmata products.


An in-depth look at Kubernetes security and compliance challenges and solutions
The 4 Kubernetes Policy Types
No Comments

Post a Comment