How to deploy RKE on vSphere with Rancher

In my previous post, I showed you how to install Rancher on k3s running on top of VMware vSphere 7. Once installed, Rancher makes it easy to deploy, run and manage Kubernetes everywhere. This post will take you to the next level and shows you how to deploy Rancher Kubernetes Engine RKE on vSphere with Rancher. Also, I’ll explain how to install and configure vSphere CSI on RKE to create and manage Kubernetes persistent volumes on vSphere.

Before we start make sure that you have a vSphere template available with a Linux OS installed and configured correctly to be used by Rancher. Of course, you can create a template manually but I recommend using Hashicorp Packer for this task. If you’re looking for Packer examples to create vSphere templates for Rancher, look here. These examples are ready to use. They need to be customized for your environment to get them working!

I used these examples also but changed the scripts a little bit to my needs. Things that I added; installing my SSH key, enable public Key authentication and install some additional packages like curl, python3, nfs-kernel-server, and docker.

In my environment, I currently have two templates available for Rancher. Both based on Ubuntu 20.04, one using DHCP and one to be used with a static IP address. In this post, I will use the template based on DHCP.

Add Cloud Credentials

First, log in to Rancher with the local admin account.

For Rancher to be able to connect to your vSphere environment a Cloud Credential is required.

Open the user menu and select Cloud Credentials.

Click Add Cloud Credential.

Enter a Name, from the dropdown menu select VMware vSphere as Cloud Credential Type, and enter your vCenter Server FQDN, Username, and Password.

Click Create.

Add a Node Template

With Rancher when you provision a Kubernetes cluster, Node Templates are used to provision the cluster nodes. These templates use Docker Machine configuration options to define an operating system image and settings/parameters for the node.

Click Manage Node Templates and then Add Template. Select vSphere. From the dropdown menu select the created vSphere Cloud Credentials.

Enter details about where the Kubernetes nodes will be deployed in your vSphere environment.

If needed, change the amount of vCPU, memory and disk size of the instance. From the dropdown menu select Deploy from template as Creation method and select your template. Click Add Network and select your VM network.

If you have tags created in your vSphere environment, those tags can be attached to the node during deployment.

Enter a name for the template. If you have Docker already installed on your template (just as I have), expand Engine Options and set Docker Install URL to none. Otherwise leave default.

Click Create.

Add an RKE template

Rancher Kubernetes Engine (RKE) is a CNCF-certified Kubernetes distribution, which is the tool that Rancher uses to provision Kubernetes clusters. When creating clusters it’s important to manage them consistently. Multi-cluster management comes with challenges to enforcing security and add-on configurations that need to be standardized before turning clusters over to end-users. RKE templates help standardize these configurations. Rancher will guarantee that every cluster it provisions from an RKE template is uniform and consistent in the way it is produced.

In the top menu, click Tools. Then select RKE Templates.

Click Add Template.

Enter a Template Name. Select Make public.

Under Kubernetes options, select External (Out-of-tree) as Cloud Provider. Leave everything else default.

Click Create.

Create a new cluster

In the top menu, select Global. Click Global then Add Cluster.

Under Create a new Kubernetes cluster select vSphere.

Enter a Cluster Name.

Also, enter a Name Prefix for the Node Pool, enter 1 for Count, from the dropdown menu select your Node Template, and select the etcd and Control Plane role.

Click Add Node Pool.

Enter a Name Prefix for the second Node Pool, enter 2 for Count, from the dropdown menu select your Node Template, enter 5 for Auto Replace, and select Drain Before Delete and the Worker role.

Select Use an existing RKE template and revision. From the dropdown menu select your RKE template.

Click Create.

After a couple of minutes, your new Kubernetes cluster will be active but not ready yet for scheduling workloads. The reason for this is that there’s no Cloud Provider installed/configured.

Install vSphere Cloud Provider

The final step is to install the vSphere Cloud Provider Interface (CPI) and Cloud Storage Interface (CSI). Installation of the vSphere CPI and CSI can be done manually but Rancher has made this very easy by providing Helm charts.

In the Rancher GUI, select your newly deployed Kubernetes cluster.

Choose Cluster Explorer from the top menu.

In Cluster Explorer, in the left top menu select Apps & Marketplace. Enter vSphere in the filter to show the available vSphere Helm charts.

First, select vSphere CPI to install. Enter your vCenter Server FQDN, Username, Password, and name(s) of your Data Center(s).

Click Install.

When the vSphere CPI is installed successfully, select vSphere CSI to install. But before you proceed make sure you have a vSphere VM Storage Policy in place to be used by the vSphere CSI. In my case, I have a policy called rancher which uses a placement rule based on a tag. This tag is attached to one of my NetApp ONTAP datastores.

Again, enter your vCenter Server FQDN, Username, Password and name(s) of your Data Center(s).

Under the CSI storage configuration, select Enable CSI Volume Resizer (vSphere 7.0U1+ required!) and enter the name of your vSphere VM Storage Policy to be used.

Click Install.

When the vSphere CSI is installed successfully, go back to Cluster Manager. In the top menu select Storage and click Storage Classes to verify the availability of the newly created vSphere Storage Class.

In the top menu select Nodes to verify that the Kubernetes cluster is now ready to schedule workloads.

Add a new workload

Let’s deploy a workload on the new Kubernetes cluster to test the vSphere Cloud Storage integration.

First, create a new Project. In the top menu select Projects/Namespaces. Click Add Project. Enter a Project Name.

Click Create.

Select the newly created Project. In the top menu select Apps. Click Launch. In the search field, enter wo and select WordPress.

Under WordPress settings, select True to Enable persistent volume for WordPress. In the dropdown menu select the vSphere Storage Class.

Under Database settings, select True to Enable persistent volume for MariaDB. In the dropdown menu select the vSphere Storage Class.

Click Launch.

To view more details about the workload, in the top menu select Resources, then click Workloads. This will show you details about the pods being provisioned for WordPress.

Select the Volumes tab to show details about the persistent volumes for WordPress.

By going to the vSphere client, selecting the datastore with the tag being used by the VM Storage Policy configured in the Storage Class, you should see two vmdk disk files which represent the persistent volumes created for WordPress.

Back in the Rancher GUI, select the Load Balancing tab. Click the WordPress xip.io link.

Et voila! Here’s the new WordPress site. Everything works as expected.

In my next blog post, I’ll dig a bit deeper into NetApp Trident CSI and how to install and configure it.