Installing the SDP

In this section, we describe how you can install SDP as a stand-alone system.

Before running the SDP, the requirements to install it need to be met.

In this page, we describe how you can install and uninstall the SDP using helm and kubectl, which do not require a clone of the SDP Integration repository.

There is also the option of using the SDP’s own Make targets, but for this you need to clone the SDP Integration repository. If you prefer to do that, please follow:

Using the SDP Makefile

In essence, both the current page and the one using Make targets describe the same steps. The former is recommended for users, while the latter is recommended for developers of the SDP, and people who are familiar with Make in general.

Note

The current page provides general instructions, most of which are cluster and environment independent. Some steps, especially gaining access to a Kubernetes cluster, will be environment specific. These are described in the relevant pages:

Helm charts

The SDP Integration repository contains two Helm charts.

The first is the main SDP chart called ska-sdp which deploys the SDP components. It is intended to be used to deploy the SDP as a subsystem of a telescope alongside other subsystems. It does not deploy the ancillary services, such as the Tango database and Kafka, which the SDP needs.

The second is an “umbrella” chart called ska-sdp-testing which is intended to deploy the SDP stand-alone for testing. It depends on the main SDP chart and other SKA charts which are used to deploy ancillary services. It is also capable of deploying Kafka.

Creating namespaces

SDP requires two namespaces, one for its control system and another for deploying its processing scripts and their execution engines (i.e. processing namespace).

If you are using a local environment, you may use the default namespace for the control system, but you will have to create a new namespace to run the processing scripts. In the commands below, we will refer to the control system namespace as <control-namespace>, and to the processing namespace as <processing-namespace>.

$ kubectl create namespace <processing-namespace>

For remote Kubernetes clusters namespaces may be pre-assigned and you won’t need to create new ones. It is important that your control and processing namespaces are different!

SDP Secrets

The SDP chart no longer creates any secret on its own, and assumes that whoever is installing the chart is going to provide the secrets.

SDP’s services can use any valid Kubernetes secret, samples for Opaque secrets are shown below.

All examples listed assume that Slurm is enabled, if it is not enabled then you can ignore those sections.

Basic Secret

Basic secrets that can be created without modifying any other configuration would be the following:

Note

All secrets listed here are fake
Secret values are base64 encoded

SDP’s Slurm’s KubeConfig secret

apiVersion: v1
kind: Secret
metadata:
  name: ska-sdp-slurm-kubeconfig
type: Opaque
data:
  kubeconfig: a3ViZWNvbmZpZw==

SDP’s Slurm’s Client secret

apiVersion: v1
kind: Secret
metadata:
  name: ska-sdp-slurm-token
type: Opaque
data:
  client_id: Y2xpZW50X2lk
  client_secret: Y2xpZW50X3NlY3JldA==

SDP’s GSM secret

apiVersion: v1
kind: Secret
metadata:
  name: ska-sdp-gsm
type: Opaque
data:
  PGHOST: cGdfaG9zdA==
  PGDATABASE: cGdfZGF0YWJhc2U=
  PGUSER: cGdfdXNlcg==
  PGPASSWORD: cGdfcGFzc3dvcmQ=
  PGSCHEMA: cGdfc2NoZW1h

Overriding Secret Names

In the case where the deployer would like to change the names in the secrets you can use the below examples:

SDP’s secrets

apiVersion: v1
kind: Secret
metadata:
  name: ska-sdp-custom-secret
type: Opaque
data:
  slurm_kubeconfig: c2x1cm1fa3ViZWNvbmZpZw==
  slurm_client_id: c2x1cm1fY2xpZW50X2lk
  slurm_client_secret: c2x1cm1fY2xpZW50X3NlY3JldA==
  gsm_pg_host: Z3NtX3BnX2hvc3Q=
  gsm_pg_db: Z3NtX3BnX2Ri
  gsm_pg_user: Z3NtX3BnX3VzZXI=
  gsm_pg_password: Z3NtX3BnX3Bhc3N3b3Jk
  gsm_pb_schema: Z3NtX3BiX3NjaGVtYQ==

Values file override

ska-sdp:
  slurmdeploy:
    enabled: true
    secrets:
      kubeconfig:
        name: ska-sdp-custom-secret
        secretName: slurm_kubeconfig
      slurmToken:
        name: ska-sdp-custom-secret
        clientIdKey: slurm_client_secret
        clientSecretKey: slurm_client_id
  gsm:
    secrets:
      name: ska-sdp-custom-secret
      databaseHost: gsm_pg_host
      databaseName: gsm_pg_db
      databaseUser: gsm_pg_user
      databasePassword: gsm_pg_password
      databaseSchema: gsm_pb_schema

Deploying the SDP

Adding the Helm repository

Releases of the SDP Helm chart are published in the SKAO central artefact repository. To install the released version, you need to add this chart repository to helm:

$ helm repo add skao https://artefact.skao.int/repository/helm-internal

If you already have the repository, you can update it, in order to gain access to latest chart versions, using:

$ helm repo update

Installing SDP with the Slurm Deployer

The Slurm Deployer was introduced in SDP 1.1.0. It is used to deploy Slurm jobs (mostly for batch processing) to a Slurm cluster. Currently, it can only connect to the AWS-based DP HPC cluster and it is disabled in the SDP chart by default. To enable its deployment, append the following --set arguments to the helm upgrade command described in the following section(s):

--set ska-sdp.slurmdeploy.enabled=true

The Slurm REST API URL has changed between versions 1.1.2 and 1.2.0, and then again later. SDP 1.3.0 contains the correct URL, which you will need to manually set for SDP versions <1.3.0:

--set slurmdeploy.slurmURL="https://api.staging.hpc.skao.int/slurm/v0.0.40" \

For SDP <1.2.0, you will also need to separately enable the vaultStaticSecret, which provide credentials for the slurm deployer to connect to the Slurm cluster. In later versions, this is automatically done for you:

--set ska-sdp.slurmdeploy.slurmURL="https://api.staging.hpc.skao.int:8080/slurm/v0.0.40" \
--set ska-sdp.slurmdeploy.vaultStaticSecret.enabled=true

Installing SDP with other complementary interfaces

There are various interfaces to SDP, which are not deployed by default. To enable their deployment, append the following --set arguments to the helm upgrade command described in the following section(s):

For ITango:

--set ska-tango-base.itango.enabled=true

For Taranta dashboards:

--set ska-tango-taranta.enabled=true --set ska-tango-tangogql.enabled=true

Enable direct reception of external data

The SDP supports receiving data from the CBF hardware, or more generally from outside the Kubernetes network, in such a way that the pods receiving the data are given direct access to the workers’ Network Interface Controller (NIC), bypassing most of the Kubernetes networking stack. When this happens, the SDP also controls the IPs that are assigned to each pod by internally managing a super-network from where sub-networks and IPs are dynamically allocated.

To achieve this, the SDP watches and reads a series of network attachment definitions (Kubernetes network-attachment-definition Custom Objects), each describing the technology used to get hold of the worker’s NIC, and the parameters used by the SDP for its internal super/sub-network management. Such network attachment definitions must be pre-defined in the underlying Kubernetes cluster, and have a series of requirements so they are usable by the SDP, namely:

They need to have a sdp.skao.int/available-for-allocation label set to true.
They need to have two annotations:
- sdp.skao.int/ip-supernet defines the super-network that the SDP will allocate sub-networks and IPs from (e.g., 192.168.1.0/24).
- sdp.skao.int/ip-subnet-cidr-bits defines the size of the individual sub-networks that are internally allocated by the SDP for specific receivers (e.g., 26).
The network attachment definition must allow the SDP to set the IP on each pod, rather than letting Kubernetes determine an IP.

By default when the SDP is deployed, it will not add the required credentials to be able to watch and read Kubernetes network attachment definitions. To enable this feature set the following option:

--set ska-sdp.helmdeploy.enableNADClusterRole=true

To get a potential list of candidate network attachment definitions usable by the SDP issue the following command:

$ kubectl get network-attachment-definitions.k8s.cni.cncf.io -A --selector=sdp.skao.int/available-for-allocation=true

Using SDP on a non-default cluster setup

On a cluster that is not using the domain of cluster.local. You will also need to set the cluster domain:

--set global.cluster_domain=<my-cluster-domain>

Installing SDP with PVC

Some of the SDP processing scripts require access to a Persistent Volume Claim (PVC) in order to store and access data. The following command asks SDP to create a PVC for you. Note that when SDP is uninstalled, the PVC is also removed.

If you already created a PVC yourself, which you want to use (independent of SDP), skip to the Installing SDP without PVC section.

Install the SDP chart with the command (assuming the release name is test). Note that we use helm upgrade --install to allow for both install and upgrade events as needed:

$ helm upgrade --install test skao/ska-sdp-testing -n <control-namespace> \
    --set ska-sdp.processingNamespace=<processing-namespace> \
    --set ska-sdp.data-pvc.create=true \
    --set ska-sdp.data-pvc.storageClassName=nfss1

The above command will create a PVC with the default name of test-pvc. You can change the name by setting global.data-product-pvc-name. ska-sdp.data-pvc.storageClassName is set to nfss1, which is used for standard SKAO managed clusters.

By default the latest version of the helm chart is deployed. If you wish to use another version, add --version <version> to the end of the helm upgrade command.

Installing SDP without PVC

If you already have a PVC you want to use, install SDP as follows. Note that we use helm upgrade --install to allow for both install and upgrade events as needed:

$ helm upgrade --install test skao/ska-sdp-testing -n <control-namespace> \
    --set global.data-product-pvc-name=<my-pvc-name> \
    --set ska-sdp.processingNamespace=<processing-namespace>

Replace <my-pvc-name> with the name of the PVC you want to use. SDP uses test-pvc by default.

By default the latest version of the helm chart is deployed. If you wish to use another version, add --version <version> to the end of the helm upgrade command.

Removing the SDP

To remove the SDP deployment from the k8s cluster, run:

$ helm uninstall test -n <control-namespace>

Remember, that if you asked SDP to create a PVC upon deployment, this command will also remove that PVC. If a pod is connected to the PVC, then the PVC will get stuck in terminating, so make sure you removed any pod that may have been using the PVC (e.g. any running processing blocks).

If you created the processing namespace, you can remove it with:

$ kubectl delete namespace <processing-namespace>