.. _installing_standalone: Installing the SDP ================== In this section, we describe how you can install SDP as a stand-alone system. Before running the SDP, the :ref:`requirements ` to install it need to be met. In this page, we describe how you can install and uninstall the SDP using helm and kubectl, which do not require a clone of the `SDP Integration repository `_. There is also the option of using the SDP's own Make targets, but for this you need to clone the SDP Integration repository. If you prefer to do that, please follow: :ref:`using-make` In essence, both the current page and the one using Make targets describe the same steps. The former is recommended for users, while the latter is recommended for developers of the SDP, and people who are familiar with Make in general. .. note:: The current page provides general instructions, most of which are cluster and environment independent. Some steps, especially gaining access to a Kubernetes cluster, will be environment specific. These are described in the relevant pages: - :ref:`remote_cluster` - :ref:`minikube_doc` Helm charts ----------- The SDP Integration repository contains two Helm charts. The first is the main SDP chart called ``ska-sdp`` which deploys the SDP components. It is intended to be used to deploy the SDP as a subsystem of a telescope alongside other subsystems. It does not deploy the ancillary services, such as the Tango database and Kafka, which the SDP needs. The second is an "umbrella" chart called ``ska-sdp-testing`` which is intended to deploy the SDP stand-alone for testing. It depends on the main SDP chart and other SKA charts which are used to deploy ancillary services. It is also capable of deploying Kafka. Creating namespaces ------------------- SDP requires two namespaces, one for its control system and another for deploying its processing scripts and their execution engines (i.e. processing namespace). If you are using a local environment, you may use the default namespace for the control system, but you will have to create a new namespace to run the processing scripts. In the commands below, we will refer to the control system namespace as ````, and to the processing namespace as ````. .. code-block:: console $ kubectl create namespace For remote Kubernetes clusters namespaces may be pre-assigned and you won't need to create new ones. It is important that your control and processing namespaces are different! .. _standalone.using-secrets: SDP Secrets ----------- The SDP chart no longer creates any secret on its own, and assumes that whoever is installing the chart is going to provide the secrets. SDP's services can use any valid Kubernetes secret, samples for Opaque secrets are shown below. All examples listed assume that Slurm is enabled, if it is not enabled then you can ignore those sections. Basic Secret ^^^^^^^^^^^^ Basic secrets that can be created without modifying any other configuration would be the following: .. note:: - All secrets listed here are fake - Secret values are base64 encoded .. code-block:: yaml :caption: SDP's Slurm's KubeConfig secret apiVersion: v1 kind: Secret metadata: name: ska-sdp-slurm-kubeconfig type: Opaque data: kubeconfig: a3ViZWNvbmZpZw== .. code-block:: yaml :caption: SDP's Slurm's Client secret apiVersion: v1 kind: Secret metadata: name: ska-sdp-slurm-token type: Opaque data: client_id: Y2xpZW50X2lk client_secret: Y2xpZW50X3NlY3JldA== .. code-block:: yaml :caption: SDP's GSM secret apiVersion: v1 kind: Secret metadata: name: ska-sdp-gsm type: Opaque data: PGHOST: cGdfaG9zdA== PGDATABASE: cGdfZGF0YWJhc2U= PGUSER: cGdfdXNlcg== PGPASSWORD: cGdfcGFzc3dvcmQ= PGSCHEMA: cGdfc2NoZW1h Overriding Secret Names ^^^^^^^^^^^^^^^^^^^^^^^ In the case where the deployer would like to change the names in the secrets you can use the below examples: .. code-block:: yaml :caption: SDP's secrets apiVersion: v1 kind: Secret metadata: name: ska-sdp-custom-secret type: Opaque data: slurm_kubeconfig: c2x1cm1fa3ViZWNvbmZpZw== slurm_client_id: c2x1cm1fY2xpZW50X2lk slurm_client_secret: c2x1cm1fY2xpZW50X3NlY3JldA== gsm_pg_host: Z3NtX3BnX2hvc3Q= gsm_pg_db: Z3NtX3BnX2Ri gsm_pg_user: Z3NtX3BnX3VzZXI= gsm_pg_password: Z3NtX3BnX3Bhc3N3b3Jk gsm_pb_schema: Z3NtX3BiX3NjaGVtYQ== .. code-block:: yaml :caption: Values file override ska-sdp: slurmdeploy: enabled: true secrets: kubeconfig: name: ska-sdp-custom-secret secretName: slurm_kubeconfig slurmToken: name: ska-sdp-custom-secret clientIdKey: slurm_client_secret clientSecretKey: slurm_client_id gsm: secrets: name: ska-sdp-custom-secret databaseHost: gsm_pg_host databaseName: gsm_pg_db databaseUser: gsm_pg_user databasePassword: gsm_pg_password databaseSchema: gsm_pb_schema Deploying the SDP ----------------- Adding the Helm repository ^^^^^^^^^^^^^^^^^^^^^^^^^^ Releases of the SDP Helm chart are published in the SKAO central artefact repository. To install the released version, you need to add this chart repository to helm: .. code-block:: console $ helm repo add skao https://artefact.skao.int/repository/helm-internal If you already have the repository, you can update it, in order to gain access to latest chart versions, using: .. code-block:: console $ helm repo update .. _inst_slurmdpl: Installing SDP with the Slurm Deployer ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The Slurm Deployer was introduced in SDP 1.1.0. It is used to deploy Slurm jobs (mostly for batch processing) to a Slurm cluster. Currently, it can only connect to the AWS-based DP HPC cluster and it is disabled in the SDP chart by default. To enable its deployment, append the following ``--set`` arguments to the ``helm upgrade`` command described in the following section(s): .. code-block:: console --set ska-sdp.slurmdeploy.enabled=true The Slurm REST API URL has changed between versions 1.1.2 and 1.2.0, and then again later. SDP 1.3.0 contains the correct URL, which you will need to manually set for SDP versions <1.3.0: .. code-block:: console --set slurmdeploy.slurmURL="https://api.staging.hpc.skao.int/slurm/v0.0.40" \ For SDP <1.2.0, you will also need to separately enable the ``vaultStaticSecret``, which provide credentials for the slurm deployer to connect to the Slurm cluster. In later versions, this is automatically done for you: .. code-block:: console --set ska-sdp.slurmdeploy.slurmURL="https://api.staging.hpc.skao.int:8080/slurm/v0.0.40" \ --set ska-sdp.slurmdeploy.vaultStaticSecret.enabled=true .. _compl_interf: Installing SDP with other complementary interfaces ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ There are various interfaces to SDP, which are not deployed by default. To enable their deployment, append the following ``--set`` arguments to the ``helm upgrade`` command described in the following section(s): For ITango: .. code-block:: console --set ska-tango-base.itango.enabled=true For Taranta dashboards: .. code-block:: console --set ska-tango-taranta.enabled=true --set ska-tango-tangogql.enabled=true Enable direct reception of external data ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The SDP supports receiving data from the CBF hardware, or more generally from outside the Kubernetes network, in such a way that the pods receiving the data are given direct access to the workers' Network Interface Controller (NIC), bypassing most of the Kubernetes networking stack. When this happens, the SDP also controls the IPs that are assigned to each pod by internally managing a super-network from where sub-networks and IPs are dynamically allocated. To achieve this, the SDP watches and reads a series of *network attachment definitions* (Kubernetes ``network-attachment-definition`` Custom Objects), each describing the technology used to get hold of the worker's NIC, and the parameters used by the SDP for its internal super/sub-network management. Such network attachment definitions must be pre-defined in the underlying Kubernetes cluster, and have a series of requirements so they are usable by the SDP, namely: - They need to have a ``sdp.skao.int/available-for-allocation`` label set to ``true``. - They need to have two annotations: - ``sdp.skao.int/ip-supernet`` defines the super-network that the SDP will allocate sub-networks and IPs from (e.g., ``192.168.1.0/24``). - ``sdp.skao.int/ip-subnet-cidr-bits`` defines the size of the individual sub-networks that are internally allocated by the SDP for specific receivers (e.g., ``26``). - The network attachment definition must allow the SDP to set the IP on each pod, rather than letting Kubernetes determine an IP. By default when the SDP is deployed, it will **not** add the required credentials to be able to watch and read Kubernetes network attachment definitions. To enable this feature set the following option: .. code-block:: console --set ska-sdp.helmdeploy.enableNADClusterRole=true To get a potential list of candidate network attachment definitions usable by the SDP issue the following command: .. code-block:: console $ kubectl get network-attachment-definitions.k8s.cni.cncf.io -A --selector=sdp.skao.int/available-for-allocation=true Using SDP on a non-default cluster setup ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ On a cluster that is not using the domain of ``cluster.local``. You will also need to set the cluster domain: .. code-block:: console --set global.cluster_domain= Installing SDP with PVC ^^^^^^^^^^^^^^^^^^^^^^^ Some of the SDP processing scripts require access to a Persistent Volume Claim (PVC) in order to store and access data. The following command asks SDP to create a PVC for you. Note that when SDP is uninstalled, the PVC is also removed. If you already created a PVC yourself, which you want to use (independent of SDP), skip to the `Installing SDP without PVC` section. Install the SDP chart with the command (assuming the release name is ``test``). Note that we use ``helm upgrade --install`` to allow for both install and upgrade events as needed: .. code-block:: console $ helm upgrade --install test skao/ska-sdp-testing -n \ --set ska-sdp.processingNamespace= \ --set ska-sdp.data-pvc.create=true \ --set ska-sdp.data-pvc.storageClassName=nfss1 The above command will create a PVC with the default name of ``test-pvc``. You can change the name by setting ``global.data-product-pvc-name``. ``ska-sdp.data-pvc.storageClassName`` is set to ``nfss1``, which is used for standard SKAO managed clusters. By default the latest version of the helm chart is deployed. If you wish to use another version, add ``--version `` to the end of the ``helm upgrade`` command. Installing SDP without PVC ^^^^^^^^^^^^^^^^^^^^^^^^^^ If you already have a PVC you want to use, install SDP as follows. Note that we use ``helm upgrade --install`` to allow for both install and upgrade events as needed: .. code-block:: console $ helm upgrade --install test skao/ska-sdp-testing -n \ --set global.data-product-pvc-name= \ --set ska-sdp.processingNamespace= Replace ```` with the name of the PVC you want to use. SDP uses ``test-pvc`` by default. By default the latest version of the helm chart is deployed. If you wish to use another version, add ``--version `` to the end of the ``helm upgrade`` command. Removing the SDP ---------------- To remove the SDP deployment from the k8s cluster, run: .. code-block:: console $ helm uninstall test -n Remember, that if you asked SDP to create a PVC upon deployment, this command will also remove that PVC. If a pod is connected to the PVC, then the PVC will get stuck in terminating, so make sure you removed any pod that may have been using the PVC (e.g. any running processing blocks). If you created the processing namespace, you can remove it with: .. code-block:: console $ kubectl delete namespace