Deploy the Kubernetes cluster

This page describes the steps required to deploy a Kubernetes cluster to the CLP.

Note

Parts of the CLP Kubernetes cluster use Ceph for the provisioning of persistent volumes. Make sure to follow Deploy the Ceph storage cluster before deploying Kubernetes.

Forming a new cluster

To form the Kubernetes cluster, start by running the following make commands:

make playbooks k8s install-base PLAYBOOKS_HOSTS=kubernetes-cluster
make playbooks k8s manual-deployment PLAYBOOKS_HOSTS=kubernetes-cluster
make playbooks k8s install TAGS=labels,taints PLAYBOOKS_HOSTS=kubernetes-cluster

Install the Kubernetes tools used to deploy workloads on the control plane:

make playbooks common k8s-tools-install PLAYBOOKS_HOSTS=kubernetes-controlplane

Installing services

Note

After the cluster is running, all services installed through playbooks only target a single control-plane node, as the resources are created on the Kubernetes API, not on the underlying nodes.

The following steps therefore set the PLAYBOOKS_HOSTS variable to kubernetes-controlplane[0]. Using any other host will probably fail, as these hosts are not equipped with the packages required to submit resources to the Kubernetes API.

Install required services

Install the services required for the Kubernetes cluster:

make playbooks k8s install TAGS=ingress,metallb,metrics,multihoming,kyverno PLAYBOOKS_HOSTS=kubernetes-controlplane[0]

Set up dynamic PersistentVolume provisioning

Dynamic provisioning of PersistentVolumes is done using Rook.io, a Kubernetes plugin that interacts with Ceph.

The Ansible collection to install this plugin uses two config files from the Ceph cluster to prevent you having to provide all of the settings manually. So, before installing Rook.io, first copy the ceph.conf and ceph.admin.client.keyring files from one of the Ceph cluster nodes to resources/ceph/:

rsync clp-k8s-worker-1a:/etc/ceph/ceph.conf resources/ceph/ceph.conf
rsync --rsync-path="sudo rsync" \
clp-k8s-worker-1a:/var/lib/ceph/<FSID>/config/ceph.client.admin.keyring \
resources/ceph/ceph.client.admin.keyring

Note

Replace <FSID> with the value found in ceph.conf

Then, install Rook.io:

make playbooks k8s install TAGS=rookio PLAYBOOKS_HOSTS=kubernetes-controlplane[0]

Set up BinderHub and JupyterHub

The following command installs BinderHub and JupyterHub:

make deploy-binderhub

Install SKAO services

This command installs the following SKAO services:

• Engineering Data Archive (EDA)

• Taranta

• TANGO Operator

make playbooks k8s install TAGS=eda,taranta,ska_tango_operator PLAYBOOKS_HOSTS=kubernetes-controlplane[0]

Adding nodes to an existing cluster

To add one or more nodes to an existing cluster, simply run the steps shown in Forming a new cluster. This will set up the new nodes and join them to the running cluster.