This is a guest blog post by Joe Mann and Daniel Paluszek at VMware.

We’ve been provided the opportunity to assess Nirmata’s integration into VMware vCloud Director and the “art of the possible” for our VMware Cloud Providers. In this guest blog post by Joe Mann and Daniel Paluszek at VMware, we will review the Nirmata integration into vCloud Director and the potential capabilities for tenants and providers.

Introduction to VMware vCloud Director

VMware vCloud Director (vCD) has had an extensive history at VMware. vCD has been traditionally focused on Infrastructure as a Service (IaaS) use cases and provides it optimally. Over the past two years, vCD has gone through a radical transformation into what we like to call a “Service Delivery Platform.” Currently, vCD has a completely new User Interface (UI) based on the Clarity HTML5 framework, multi-site capabilities, an appliance form-factor, and several service integrations from ecosystem partners like Nirmata.

In the context of vCD, an organization is synonymous with a tenant. vCD provides a cloud management plane that abstracts the underlying platform in terms of virtual data centers: provider virtual data centers (pVDC) and organization virtual data centers (oVDC).

By utilizing these tenancy constructs, vCD can instantiate and adhere to secure, multi-tenant operating parameters to provide secure resource allocation. This can be accessed by the organization or provider via the native user interface (UI) or the extensible API.

vCD 10.0 is our newest release of this comprehensive service delivery platform. You can read more about this exciting release on Daniel’s blog site (Part 1 and Part 2) or watch our lightboard video on YouTube.

Container Service Extension

The Container Service Extension (CSE) is a VMware vCloud Director (vCD) extension that helps tenants create, lifecycle manage, and interact with Kubernetes clusters in vCD-managed environments. There are currently two versions of CSE: Standard and Enterprise.

However, when utilizing vCD, it’s not an either-or, but a provider could productize both permutations inside of their architecture. The below diagram depicts how one could co-exist between the two versions along with resources backed by vCenter, NSX-V, and NSX-T.

CSE Standard brings Kubernetes-as-a-Service to vCD by creating customized VM templates and enabling tenants to deploy fully functional Kubernetes clusters in self-contained vApps.

CSE Enterprise introduces the ability of the CSE server to utilize a service account to communicate directly with the PKS and NSX-T APIs to automate the deployment of VMware Enterprise PKS Kubernetes clusters and its supporting networking resources in a vCD managed environment.

Enterprise PKS brings additional, enterprise-grade features and functionality in addition to what is provided with CSE Standard Kubernetes clusters. This includes, but is not limited to:

• HA, multi-master Kubernetes clusters
• Dynamic persistent storage provisioning with the vSphere Cloud Provider integration
• Automated Day 1 and Day 2 Kubernetes cluster management via Bosh Director
• Microsegmentation capability for Kubernetes resources via integration with NSX-T
• Automated creation of Kubernetes service type “LoadBalancer” and ingress resources via NSX-T L4/L7 load balancers
• Support for Harbor, an open-source, cloud-native registry

To learn more about CSE and how to deploy the service, refer to the installation documentation as well as Joe’s blog series on deployment workflow.

Nirmata’s vCD Integration

One of the main tenants of Nirmata’s mission statement is to make it as simple as possible for users to deploy infrastructure to support Cloud Native workloads. VMware has designed our solutions to provide that same simplicity for our Service Provider community with the introduction of CSE for vCD. With CSE, tenants are able to provision clusters with a single command using the vcd-cli, a command-line interface for vCD.

However, for those vCD tenants new to the world of Kubernetes, the command line can be a bit intimidating. That’s where Nirmata comes in! With the Nirmata Kubernetes Extension for vCloud Director, tenants can provision Kubernetes clusters directly from the vCD tenant portal user interface with a single click! Nirmata makes this extremely intuitive by allowing the organization users to create their own cluster while specifying available networks, storage policies, and distinct Nirmata policies.

Since the vCloud Director 9.1 release, VMware has offered users the ability to extend the user interface of vCD with custom extensions, which gives service providers and partners like Nirmata the ability to add their own UI elements and workflows directly inside the vCloud Director HTML5 client. Find out more about the vcd-ext-sdk template on our GitHub page.

The Nirmata Kubernetes Extension uses this extensibility functionality to communicate directly with the CSE Server to orchestrate the provisioning of Kubernetes clusters within a tenants’ OrgVDC.

Not only does the Nirmata Kubernetes Extension allow for net-new Kubernetes provisioning and management, users can also use the extension to manage existing Kubernetes clusters provisioned via vCD. Upon creation or registration for existing clusters, the Nirmata controller is automatically deployed to the cluster to allow for advanced observability and management of the clusters directly from the vCD tenant portal. The Nirmata Kubernetes Extension also provides enhanced visibility of Enterprise PKS Kubernetes clusters provisioned via CSE Enterprise. Now, tenants can consume detailed information about their CSE Standard and Enterprise clusters natively within the vCD tenant portal.

The Nirmata Kubernetes Extension also brings a litany of Day 2 operations capabilities that many end-users will require in a Kubernetes-as-a-Service offering powered by vCD. For instance, Nirmata provides a set of curated “add-ons” such as Prometheus (for advanced monitoring capabilities) and Velero (for advanced backup and recovery for Kubernetes resources) that can be deployed to a tenant’s cluster through the vCD UI extension. Service Providers can also create their own, custom add-ons and curate those add-ons into a profile that can be applied to a cluster upon creation.

For example, a service provider can build a profile that includes Prometheus, Velero, and NGINX services and assign that profile to a tenant group in vCD. An end-user within that tenant can choose to utilize this profile when deploying their cluster via the vCD tenant portal and these services associated with the profile will be deployed automatically on the Kubernetes cluster upon provisioning.

High-Level Deployment Steps

Summary of Deployment Steps:

1. Establish Container Service Extension (CSE) server that will co-exist with vCD and an existing RabbitMQ server for communication exchange.
2. Deployment of Nirmata Docker container.
3. Propagation of Nirmata vCD UI plugin and API integration.
4. Configuration of Nirmata vCD UI plugin to Nirmata SaaS platform.

Before we can set up Nirmata for a vCD environment, we need to prepare our CSE environment first. This requires deploying the CSE server and granting tenants within vCD the specific rights required to deploy Kubernetes clusters, as CSE utilizes the existing Identity Providers configuring within vCD to enforce RBAC for cluster provisioning. As discussed previously, Joe did a great job of documenting the steps A RabbitMQ server is also required for both CSE and the Nirmata Kubernetes Extension as both services use RabbitMQ exchanges to communicate with each other and vCD.

Once CSE has been deployed in the vCD environment, the Nirmata server and client extensions can be deployed. The Nirmata server is delivered as a docker container that can be deployed on any server within the vCD network with the docker engine service available.

 

The Nirmata server component must be able to communicate with the CSE server, vCD cells, and the RabbitMQ server. The docker container is deployed utilizing a configuration file that contains access information for the vCD and RabbitMQ servers, which allows the Nirmata server to communicate directly with both services.

When the docker container is deployed, the Nirmata server uses the RabbitMQ access information from the configuration file to automatically create the `nirmataext` exchange required for sending messages between CSE, vCD, and the Nirmata server. As shown below, we can tail the Nirmata container log to see exchange queue messages that are relayed from RabbitMQ.

Once the Nirmata server extension has been installed, the Nirmata client extension and UI plugin can be configured. After obtaining the Nirmata plugin .zip file, it can be uploaded to vCD and published to a specific set of tenants.

 

This allows system admins another layer of control over what tenants have the access to provision Kubernetes clusters. After installing the UI extension, the API extensions need to be registered with vCD via the vcd-cli.

The Nirmata server receives requests initiated by the tenant users from the vCD client extension and initiates the deployment of Kubernetes clusters via CSE as well as instantiating the Nirmata controller deployment on newly created (or existing) clusters. The Nirmata controller passes cluster statistics back to the Nirmata SaaS and vCD platforms. Last of all, this allows users to deploy additional services (add-ons) to the clusters from directly within the vCD portal.

Evaluation Results

Overall, we are impressed with the ease of use and the intuitive experience of Nirmata with vCD. Once the provider administrator establishes all necessary components for self-serviceability, a tenant user can quickly operationalize a cluster within the context of vCD.

 

We also enjoy seeing the value of Nirmata’s SaaS platform and the value add it brings to tenants and providers. From the picture below, one can see the availability of this cluster along with specific details such as availability, alarms, and detailed component view.

Not only can the Nirmata Extension deploy and manage net-new clusters, but tenant users can also register existing clusters provisioned outside of the Nirmata Extension:

Registering an existing cluster allows the Nirmata server to deploy the “nirmata-kube-controller” to the Kubernetes cluster.

The controller passes information back to the vCD UI and Nirmata SaaS platform to provide the same visibility and control over the cluster as we’d get if we provisioned the cluster directly through the extension’s UI, as we can see below:

 

In Daniel’s evaluation, the Nirmata solution for vCD brings an intuitive experience for self-serviceability. While the Container Service Extension is fairly new and has evolved since inception, I see significant value of Nirmata in a VMware Cloud Provider Platform, even with organizations that are just starting with a Kubernetes strategy. Most importantly, we now have alignment of VM and Kubernetes-based workloads in the same cloud management platform: VMware vCloud Director.

Summary

Nirmata brings several additional value points for onboarding a container strategy with VMware vCloud Director. As stated above, the ability to co-exist between CSE Standard, CSE Enterprise, and traditional VM workloads in the same UI is unparalleled as of today.

The Container Service Extension, along with the Nirmata Kubernetes Extension for VMware vCloud Director, creates the foundation for a Service Provider to productize an advanced Kubernetes-as-a-Service offering for their tenants to consume via vCD.

To get started with the Nirmata Kubernetes Extension for vCloud Director, contact us to receive additional information on the overall solution and check out the demo video below

 

About the Authors:

Joe Mann is a Staff Cloud Solutions Architect at VMware who is part of the World Wide Cloud Practice team. Joe’s focus within the WWCP team is helping build out the Cloud Native Strategy, on a global level, for the VMware Cloud Provider Program. Check out updates from Joe on all things Cloud-Native at VMware at his blog (https://mannimal.blog) and follow along on Twitter at @vmmannimal

Daniel Paluszek is a Staff Cloud Solutions Engineer at VMware in the Cloud Provider Program supporting strategic cloud service providers. Daniel has been in the IT industry for 15+ years while starting as a VMware customer. He is passionate about learning and collaborating with customers on solutions. Daniel publishes at https://www.paluszek.com and https://blogs.vmware.com/cloudprovider. Follow Daniel on Twitter @dpaluszek.