RKE on vSphereIn my previous post, I showed you how to install Rancher on k3s running on top of VMware vSphere 7. Once installed, Rancher makes it easy to deploy, run and manage Kubernetes everywhere. This post will take you to the next level and shows you how to deploy Rancher Kubernetes Engine RKE on vSphere with Rancher. Also, I’ll explain how to install and configure vSphere CSI on RKE to create and manage Kubernetes persistent volumes on vSphere.

Before we start make sure that you have a vSphere template available with a Linux OS installed and configured correctly to be used by Rancher. Of course, you can create a template manually but I recommend using Hashicorp Packer for this task. If you’re looking for Packer examples to create vSphere templates for Rancher, look here. These examples are ready to use. They need to be customized for your environment to get them working!

I used these examples also but changed the scripts a little bit to my needs. Things that I added; installing my SSH key, enable public Key authentication and install some additional packages like curl, python3, nfs-kernel-server, and docker.

In my environment, I currently have two templates available for Rancher. Both based on Ubuntu 20.04, one using DHCP and one to be used with a static IP address. In this post, I will use the template based on DHCP.

RKE on vSphere with Rancher

Add Cloud Credentials

First, log in to Rancher with the local admin account.

RKE on vSphere with Rancher

For Rancher to be able to connect to your vSphere environment a Cloud Credential is required.

Open the user menu and select Cloud Credentials.

RKE on vSphere with Rancher

Click Add Cloud Credential.

Enter a Name, from the dropdown menu select VMware vSphere as Cloud Credential Type, and enter your vCenter Server FQDN, Username, and Password.

RKE on vSphere with Rancher

Click Create.

RKE on vSphere with Rancher

Add a Node Template

With Rancher when you provision a Kubernetes cluster, Node Templates are used to provision the cluster nodes. These templates use Docker Machine configuration options to define an operating system image and settings/parameters for the node.

Click Manage Node Templates and then Add Template. Select vSphere. From the dropdown menu select the created vSphere Cloud Credentials.

RKE on vSphere with Rancher

Enter details about where the Kubernetes nodes will be deployed in your vSphere environment.

RKE on vSphere with Rancher

If needed, change the amount of vCPU, memory and disk size of the instance. From the dropdown menu select Deploy from template as Creation method and select your template. Click Add Network and select your VM network.

RKE on vSphere with Rancher

If you have tags created in your vSphere environment, those tags can be attached to the node during deployment.

RKE on vSphere with Rancher

Enter a name for the template. If you have Docker already installed on your template (just as I have), expand Engine Options and set Docker Install URL to none. Otherwise leave default.

RKE on vSphere with RancherClick Create.RKE on vSphere with Rancher

Add an RKE template

Rancher Kubernetes Engine (RKE) is a CNCF-certified Kubernetes distribution, which is the tool that Rancher uses to provision Kubernetes clusters. When creating clusters it’s important to manage them consistently. Multi-cluster management comes with challenges to enforcing security and add-on configurations that need to be standardized before turning clusters over to end-users. RKE templates help standardize these configurations. Rancher will guarantee that every cluster it provisions from an RKE template is uniform and consistent in the way it is produced.

In the top menu, click Tools. Then select RKE Templates.

Click Add Template.

RKE on vSphere with Rancher

Enter a Template Name. Select Make public.

RKE on vSphere with Rancher

Under Kubernetes options, select External (Out-of-tree) as Cloud Provider. Leave everything else default.

RKE on vSphere with Rancher

Click Create.

RKE on vSphere with Rancher

Create a new cluster

In the top menu, select Global. Click Global then Add Cluster.

Under Create a new Kubernetes cluster select vSphere.

RKE on vSphere with Rancher

Enter a Cluster Name.

Also, enter a Name Prefix for the Node Pool, enter 1 for Count, from the dropdown menu select your Node Template, and select the etcd and Control Plane role.

Click Add Node Pool.

Enter a Name Prefix for the second Node Pool, enter 2 for Count, from the dropdown menu select your Node Template, enter 5 for Auto Replace, and select Drain Before Delete and the Worker role.

Select Use an existing RKE template and revision. From the dropdown menu select your RKE template.

RKE on vSphere with Rancher

Click Create.

RKE on vSphere with Rancher

RKE on vSphere with Rancher

After a couple of minutes, your new Kubernetes cluster will be active but not ready yet for scheduling workloads. The reason for this is that there’s no Cloud Provider installed/configured.

RKE on vSphere with Rancher

Install vSphere Cloud Provider

The final step is to install the vSphere Cloud Provider Interface (CPI) and Cloud Storage Interface (CSI). Installation of the vSphere CPI and CSI can be done manually but Rancher has made this very easy by providing Helm charts.

In the Rancher GUI, select your newly deployed Kubernetes cluster.

Choose Cluster Explorer from the top menu.

In Cluster Explorer, in the left top menu select Apps & Marketplace. Enter vSphere in the filter to show the available vSphere Helm charts.

RKE on vSphere with Rancher

First, select vSphere CPI to install. Enter your vCenter Server FQDN, Username, Password, and name(s) of your Data Center(s).

RKE on vSphere with Rancher

Click Install.

RKE on vSphere with Rancher

When the vSphere CPI is installed successfully, select vSphere CSI to install. But before you proceed make sure you have a vSphere VM Storage Policy in place to be used by the vSphere CSI. In my case, I have a policy called rancher which uses a placement rule based on a tag. This tag is attached to one of my NetApp ONTAP datastores.

RKE on vSphere with Rancher

Again, enter your vCenter Server FQDN, Username, Password and name(s) of your Data Center(s).

RKE on vSphere with Rancher

Under the CSI storage configuration, select Enable CSI Volume Resizer (vSphere 7.0U1+ required!) and enter the name of your vSphere VM Storage Policy to be used.

RKE on vSphere with Rancher

Click Install.

RKE on vSphere with Rancher

When the vSphere CSI is installed successfully, go back to Cluster Manager. In the top menu select Storage and click Storage Classes to verify the availability of the newly created vSphere Storage Class.

RKE on vSphere with Rancher

In the top menu select Nodes to verify that the Kubernetes cluster is now ready to schedule workloads.

RKE on vSphere with Rancher

Add a new workload

Let’s deploy a workload on the new Kubernetes cluster to test the vSphere Cloud Storage integration.

First, create a new Project. In the top menu select Projects/Namespaces. Click Add Project. Enter a Project Name.

Click Create.

RKE on vSphere with Rancher

Select the newly created Project. In the top menu select Apps. Click Launch. In the search field, enter wo and select WordPress.

RKE on vSphere with Rancher

Under WordPress settings, select True to Enable persistent volume for WordPress. In the dropdown menu select the vSphere Storage Class.

RKE on vSphere with Rancher

Under Database settings, select True to Enable persistent volume for MariaDB. In the dropdown menu select the vSphere Storage Class.

RKE on vSphere with Rancher

Click Launch.

RKE on vSphere with Rancher

To view more details about the workload, in the top menu select Resources, then click Workloads. This will show you details about the pods being provisioned for WordPress.

RKE on vSphere with Rancher

Select the Volumes tab to show details about the persistent volumes for WordPress.

RKE on vSphere with Rancher

By going to the vSphere client, selecting the datastore with the tag being used by the VM Storage Policy configured in the Storage Class, you should see two vmdk disk files which represent the persistent volumes created for WordPress.

RKE on vSphere with Rancher

Back in the Rancher GUI, select the Load Balancing tab. Click the WordPress xip.io link.

RKE on vSphere with Rancher

Et voila! Here’s the new WordPress site. Everything works as expected.

RKE on vSphere with Rancher

In my next blog post, I’ll dig a bit deeper into NetApp Trident CSI and how to install and configure it.