General guidelines
==================

The SDP Integration repository integrates the components of
the SKA Science Data Processor (SDP).

It is a public GitLab project accessible from::

    https://gitlab.com/ska-telescope/sdp/ska-sdp-integration

Cloning the repository
----------------------

You may clone it via SSH or HTTPS.
For SSH:

.. code-block:: console

    $ git clone git@gitlab.com:ska-telescope/sdp/ska-sdp-integration.git --recurse-submodules

And for HTTPS:

.. code-block:: console

    $ git clone https://gitlab.com/ska-telescope/sdp/ska-sdp-integration.git --recurse-submodules

The ``--recurse-submodules`` option makes sure that you clone the
`ska-cicd-makefile <https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile>`_
submodule together with the repository, which is an important part for development.

.. note::

    Make sure you include ``--recurse-submodules`` when you run ``git pull`` as well,
    to keep up-to-date with the submodule too.

Branch-based development
------------------------

We follow a branch-based development process. New code is always added to a
branch created from ``master``. Then a (draft) Merge Request is created,
which needs to be reviewed and approved by ``CODEOWNERS`` before it can be merged.
Only maintainers of the repository can merge code to master. Only code on
the master branch is released.

Using poetry
------------

Although the repository does not contain a Python package (its main artefact
is the Helm chart), its tests are written in Python. The project uses
`Poetry <https://python-poetry.org/docs/>`_ for package management.
Make sure you have it installed and configured to your needs. This is
needed if you are developing and running tests.

Python dependencies of the project are collected in ``pyproject.toml``.
A ``poetry.lock`` file is also included, which pins the versions of
dependencies the code has been tested with. If you change the
requirements in ``pyproject.toml`` make sure you also update the ``poetry.lock`` file.

The dependencies are organised into four groups in the ``pyproject.toml`` file:

    - ``test`` for running the tests
    - ``test-dev`` for developing the tests (formatting and linting utilities)
    - ``docs`` for producing the documentation

Repository structure
--------------------

The main layout of the repository is as follows.

The most important top level files include:

    - ``.gitlab-ci.yml``
    - ``.readthedocs.yaml``
    - ``CHANGELOG.rst,``
    - ``CODEOWNERS``
    - ``CONTRIBUTORS``
    - ``LICENCE``
    - ``Makefile``
    - ``README.rst``
    - ``poetry.lock``
    - ``pyproject.toml``

When developing the code, update these as needed.

Directory structure:

.. list-table::
   :header-rows: 1

   * - Folder
     - Description
   * - ``.gitlab/ci``
     - Contains YAML files for GitLab CI/CD job definitions
   * - ``.make``
     - Contains the ska-cicd-makefile submodule (not directly changeable from the SDP repository)
   * - ``charts``
     - Helm charts. The ``ska-sdp`` chart is the main product of the repository.
       Includes data like EDA configuration and Taranta dashboard files.
   * - ``docs``
     - Documentation and related content
   * - ``resources/values``
     - Values files that can be used to customise the SDP installation
   * - ``tests``
     - All of the tests and test-related infrastructure

Documentation
-------------

We document using
`Sphinx and Read The Docs <https://docs.readthedocs.io/en/stable/intro/getting-started-with-sphinx.html>`_.
Every time you change the code in the repository, make sure you
review the documentation, and update it as needed.