Traditionally, containers have been difficult to work with. But during the last years, containers have become increasingly common and crucial in cloud computing. And when you say ‘Containers’, Kubernetes (or K8s) is never far away. Kubernetes is a system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery. Kubernetes was originally developed by Google and it quickly became the leading product in its space. But this added another layer of complexity.

Pivotal Container Service (PKS) was announced at VMworld 2017 and customers have been waiting anxiously. Now Pivotal Container Service (PKS) became generally available.

But what is Pivotal Container Service (PKS)?

Well, it’s an enterprise-class platform for creating, deploying and running Kubernetes clusters for both development and production. But that’s the marketing story.

What is it really?!

Pivotal Container Service (PKS) is a combination of VMware, Pivotal and Kubernetes that enables enterprises and service providers to deliver production-ready Kubernetes on VMware vSphere and Google Cloud Platform (GCP), with constant compatibility to Google Container Engine (GKE).

The most important initial capabilities of PKS 1.0 are:

  • Fully supported Kubernetes distribution.
  • BOSH orchestrated easy deployment with integrated lifecycle management and operations.
  • Multi-cloud capabilities with initial support on vSphere and GCP.
  • Support for Kubernetes 1.9.2, the latest stable version of Kubernetes.
  • Advanced container networking with NSX-T.
  • Enterprise-grade security with isolation, policies, vulnerability scanning, and content trust.
  • Multi-tenancy with cluster-level security and autonomy.
  • Rapid, on-demand provisioning of Kubernetes clusters.
  • Production-grade features such as high availability, security, identity and access management, monitoring and logging at both the infrastructure and application layer.
Self-service

The primairy goal of PKS is to simplify the deployment of Kubernetes and run containers on vSphere and GCP. PKS provides a simplified set of APIs and a CLI that lets platform operators automate the deployment of Kubernetes clusters. This includes complex tasks such as configuring and provisioning load balancers, networks, and security policies. PKS enables developers to launch, scale, and interact with their own Kubernetes clusters by using the familiar Kubernetes APIs or kubectl commands.

Multitenancy

Multiple Kubernetes clusters can be deployed and managed in a multitenant way from a single control plane. To achieve this, PKS uses VMware NSX-T to isolate the clusters using NSX security policies. Kubernetes clusters can be deployed into different vSphere clusters and configured to use different datastores. The result is complete isolation and the avoidance of noisy/nosey neighbour problem between tenants.

Security

Containers and security has always been an issue. PKS includes micro-segmentation, security policies, container image signing, vulnerability scanning, and user identity and access management through User Account and Authorization Service (UAA). All with the goal to deliver a highly secure solution. PKS enables the automated delivery of network topologies and micro-segmentation to each container in the Kubernetes cluster through the integration with NSX-T. User Account and Authorization Service (UAA) lets admins use their enterprise credentials to access the PKS control plane. In addition, PKS scans images for vulnerabilities, signs and verifies images, and provides auditing capabilities for enterprise security and compliance.

Image management

Containers cannot live without a container registry. So, PKS includes Project Harbor which is an open source enterprise container registry. Harbor simplifies image management with distribution, replication and security mechanisms. The Harbor registry uses a logical construct called Project. This is used to group users and repositories to enable fine-grained access control.

Self healing

One of the coolest features is the self healing capability of the solution. BOSH is the vital component within the PKS solution which delivers just that. It monitors the health of clusters and enables self-healing to enable clusters to run at optimal capacity. If BOSH deems a node unhealthy, PKS automatically detects its state and resurrects it without workload downtime. In addition, patching and upgrades of Kubernetes nodes can be managed from the PKS platform in a centralized fashion, without impact to running applications.

One of the most compelling parts of this offering is that customers are guaranteed to always have the latest Kubernetes version available to them. PKS maintains constant compatibility to Google Container Engine (GKE) for the latest Kubernetes releases.

Components that make up PKS

Now that we know what PKS can deliver, the techie in me wants to know what’s under the hood. Well, the foundation is VMware which functions as the overall management layer handling security and networking among other tasks. On top of that Pivotal add the infrastructure piece. The last piece is Kubernetes to orchestrate containers at the enterprise level.

PKS ships as a standalone product, and integrates with a customer’s vSphere infrastructure. PKS is provided as an OVA.

PKS consists of the following components:

  • Kubo – Kubernetes provisioned via BOSH.
  • NSX-T – networking and security for containers.
  • GCP Service Broker – to integrate Google Cloud Services into applications.
  • vSphere Cloud Provider – to enable persistent storage for stateful applications.
  • Harbor – Container Repository which also includes vulnerability checking, and image signing and verification.

 

Pivotal Container Service

 Kubo

PKS uses a Pivotal feature called BOSH which enables users to provision Kubernetes onto their on-premises vSphere environment. This provisioning capability is named ‘Kubo‘. Kubo is a joint project between Google and Pivotal which allows for the creation of Kubernetes clusters on multiple different platforms. PKS only uses this functionality and does not require a full-blown Pivotal Cloud Foundry (PCF) deployment. PKS only uses the BOSH feature.

Besides provisioning BOSH also provides monitoring and self-healing for Kubernetes, making the Kubernetes clusters that it provisions highly available.

NSX-T

PKS includes VMware NSX-T, which offers advanced container networking and security features for Kubernetes clusters. NSX-T provides the complete set of Layer 2 through Layer 7 networking services that are needed for containers and pod-level networking. With NSX-T integration in PKS, enterprises will be able to quickly deploy networks with micro-segmentation and on-demand network virtualization without disrupting the development cycle.

With NSX-T, customers get all the networking functions required for Kubernetes, including pod-level networking, ingress to services, and load balancing across multiple replica sets. In addition to the basic Kubernetes networking functions, customers get advanced networking functions, such as network security policies and tenant-level isolation using the NSX-T multi-tiered routing model.

 

A key design concept of NSX-T integration with PKS is to assign a unique logical switch to each Kubernetes namespace. This provides the ability to segment the traffic of each namespace within a given Kubernetes cluster. Development teams will be able to choose to use a dedicated Kubernetes namespace within a shared cluster to secure their workloads from other teams.

GCP Service Broker

PKS includes a service broker that provides out-of-the-box access to GCP services.  It enables an operator to expose selected GCP services so that development teams can provision and consume GCP services by creating and managing service instances with the kubectl CLI or API. The GCP service broker supports offering GCP subscription services such as Google Cloud Storage, Google BigQuery, and Google Stackdriver. These services will be able to be consumed by applications running on-premises or from within GCP.

vSphere Cloud Provider

PKS enables you to deploy Kubernetes clusters for both stateless and stateful applications. It supports the VMware vSphere Storage for Kubernetes plugin which is part of Kubernetes through Project Hatchway. The plugin allows PKS to support Kubernetes storage primitives on vSphere storage; the storage primitives include volumes, persistent volumes, persistent volumes claims, storage classes, and stateful sets. The storage plugin also brings in enterprise-grade storage features. By using VMware vSAN, for example, you can extend storage policy-based management to applications running in a Kubernetes cluster.

Harbor

Harbor is an open source enterprise-class container registry server that stores and distributes container images. It provides production-grade authentication and role-based access to push and pull images. It also provides key registry services, such as integrated vulnerability scanning, image trust services, and image replication services.

With Harbor, container images can be safely and securely downloaded into Kubernetes clusters for application deployment. The Harbor registry enables production-grade image repositories for CI/CD pipelines. Customers can safely push container images into Harbor as part of their application release automation process. These images can be scanned for vulnerabilities and have their signatures validated by Harbor before they are allowed to be pulled into Kubernetes clusters as part of an application workload deployment process.

This gives development teams the platform to deploy applications quickly while still giving IT the control to enable the container images meet the security requirements of the enterprise.