.. _using-make:

Using the SDP Makefile
======================

The ska-sdp-integration repository contains a
`Makefile <https://gitlab.com/ska-telescope/sdp/ska-sdp-integration/-/blob/master/Makefile>`_
which provides targets to simplify deploying and testing the SDP.
The following commands should be run from the root directory of
the git repository.

If you want to install without access to the repository, using helm and kubectl,
follow the instructions here: :ref:`Installing the SDP stand-alone <installing_standalone>`

This page provides general instructions, most of which are cluster
and environment independent. For environment-specific differences see the relevant pages:

  - :ref:`remote_cluster`
  - :ref:`minikube_doc`

Environment variables
---------------------

The environment variables to control the deployment and testing of the SDP are
defined at the top of the Makefile. They are described in the table below.

.. list-table::
  :widths: auto
  :header-rows: 1

  * - Environment variable
    - Default value
    - Description
  * - ``KUBE_NAMESPACE``
    - ``test``
    - Kubernetes namespace for deploying the control system
  * - ``KUBE_NAMESPACE_SDP``
    - ``test-sdp``
    - Kubernetes namespace for deploying the processing scripts
  * - ``CHART_RELEASE``
    - ``test``
    - Name of the Helm chart release
  * - ``CHART_REPO``
    - ``charts``
    - Location of the Helm chart repository, may be a directory or name of a remote repository
  * - ``CHART_VERSION``
    - empty
    - Version of the chart to install (useful when using a remote repository)
  * - ``CHART_VALUES``
    - ``resources/values/test-external.yaml``
    - Values file(s) to use to install/upgrade the SDP deployment, may be a list of files separated by spaces
  * - ``HELM_TIMEOUT``
    - ``5m``
    - Timeout for `helm upgrade` and `helm install`
  * - ``HELM_UNINSTALL_HAS_WAIT``
    - ``"1"``
    - Whether ``helm uninstall`` supports the ``--wait`` flag
  * - ``TEST_INGRESS``
    - Not set
    - Ingress to use to execute tests
  * - ``TEST_TANGO_CLIENT``
    - ``gql``
    - Client to use to connect to Tango devices in the tests
  * - ``TEST_MARKER``
    - ``"not extended_visibility_receive_test"``
    - Pytest markers to be used in the tests
  * - ``PYTEST_FLAGS``
    - ``""``
    - Additional flags passed to ``pytest``

The default values are defined to be suitable for testing in Minikube.

If you are using different values from the default ones, you will have to
export the environment variables, or use them together with the commands.
Alternatively, you can temporarily update the Makefile, but make sure
you are working from a branch so that you don't accidentally commit your changes.

For example:

.. code-block:: console

    $ export KUBE_NAMESPACE=default
    $ export KUBE_NAMESPACE_SDP=sdp

    $ make install-sdp

The above will install SDP into the ``default`` namespace and use the ``sdp`` namespace
for processing script deployments.

Creating namespaces
-------------------

Create the namespaces specified in the environment variables with:

.. code-block:: console

    $ make create-namespaces

Chart dependencies
------------------

If you are using the local copy of the chart to install SDP, you can make sure
you have the latest chart dependencies with:

.. code-block:: console

    $ make build-chart-dependencies

which will download the dependencies listed in the ``Chart.lock`` file. Note
that Helm will not download dependencies from arbitrary URLs; it must know
about the repositories in advance. You may need to add the repositories with
the ``helm repo add`` command, e.g.

.. code-block:: console

    $ helm repo add skao https://artefact.skao.int/repository/helm-internal
    $ helm repo add bitnami https://charts.bitnami.com/bitnami

If you are developing the chart and want to update the dependencies and rebuild
the ``Chart.lock`` file, then you should run:

.. code-block:: console

    $ make update-chart-dependencies

Using a custom values file
--------------------------

You can specify a custom values.yaml file to be used when SDP is installed,
using the ``CHART_VALUES`` environment variable. In this, you can enable
or disable components, and set the parameters for the Persistent
Volume Claim (PVC) to be used for data processing.

The following example will deploy ITango, and will ask SDP to create a PVC
with a given storage class:

.. code-block::

    ska-tango-base:
      itango:
        enabled: true

    ska-sdp:
      data-pvc:
        create:
          enabled: true
          storageClassName: nfss1
          size: 5Gi

Assuming you save the above into ``my_values.yaml``, export it as follows to be used:

.. code-block:: console

    $ export CHART_VALUES=my_values.yaml

There are various custom values files provided in the
`resources/values <https://gitlab.com/ska-telescope/sdp/ska-sdp-integration/-/tree/master/resources/values>`_
directory of the ska-sdp-integration repository.

Deploying the SDP
-----------------

.. code-block:: console

    $ make install-sdp

By default, the SDP will be installed using the local copy of the Helm chart in
the ``charts`` directory.

If instead you want to use the published version of the SDP chart from the SKA
Central Artefact Repository, then you can do that using the ``CHART_REPO``
variable. First, add the Helm chart repository:

.. code-block:: console

    $ helm repo add skao https://artefact.skao.int/repository/helm-internal

Then install SDP using this repository:

.. code-block:: console

    $ make install-sdp CHART_REPO=skao

The ``install-sdp`` command upgrades the deployment if it exists,
or installs it if it doesn't. It has a ``--wait`` switch added to it,
which means the command will only return once the deployment has
finished installing/upgrading.

Cleaning up and removing SDP
----------------------------

Once you finished work with SDP, you can use the following
commands to clean up your deployment.

Uninstall SDP:

.. code-block:: console

    $ make uninstall-sdp

Delete namespaces (optional, only if you created the namespaces)

.. code-block:: console

    $ make delete-namespaces