.. _scheduled_tests:

CI scheduled builds
===================

``skart`` can be used as a tool to check whether a repository
continues to pass its unit tests when dependent components
such as Python packages, OCI image builds, or dependent
chart releases are updated. Either released or development
versions of these components can be selected for this testing.
This gives early indication of any issues which may need
to be resolved before these components are released or developed
further.

The :ref:`skart_file` section describes how to set up ``skart`` to work
in a given repository. This is also prerequisite to run scheduled
updates with ``skart``.

It is often useful to automatically run a Gitlab scheduled
CI pipeline which uses ``skart`` to update the components used
before continuing to its normal build and test stages.
This GitLab scheduled pipeline can be run at
appropriate time(s) of day (often overnight) and with a suitable CI
variable set to trigger the ``skart`` stage.

Note that this will require all Gitlab repositories referenced in
``skart.toml`` to be configured such that they allow access by the
scheduled CI jobs. To ensure this, update "Job token permissions" in
the "CI/CD" settings of the referenced repository so that they allow
access from the repository you are adding the scheduled run to.

As a working example this is the start of the file
``.gitlab-ci.yml`` from
the repository
`ska-sdp-proccontrol <https://gitlab.com/ska-telescope/sdp/ska-sdp-proccontrol>`_:

  .. code:: yaml

    skart-update:
      stage: .pre
      tags:
        - ska-default
      rules:
        - if: '$DO_SKART_UPDATE'
          when:  on_success
        - when: never
      before_script:
        - pip3 install --upgrade --index-url=https://artefact.skao.int/repository/pypi-internal/simple skart
        - poetry config virtualenvs.create false
      script:
        - make deps-update
      artifacts:
        paths:
          - poetry.lock
          - pyproject.toml

This specific example uses the ``deps-update`` make target from
`SKA CI/CD Makefiles <https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile/-/blob/master/dependencies.mk?ref_type=heads#L49>`_,
which executes ``skart update`` followed by
``poetry update --lock``. Alternatively, one can use the
`deps-update-release <https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile/-/blob/master/dependencies.mk?ref_type=heads#L77>`_
make target to use ``skart`` to update with released versions of dependencies.

It can be run on schedule, but also as part of any pipeline, in this
specific case, if the ``DO_SKART_UPDATE`` CI variable is set.
After this ``.pre`` stage, the CI pipeline continues as normal
through its stages (including unit testing) against any updated artefacts.

In the ``artifacts/paths`` section of the CI job, we save files that
have been updated by ``skart`` and/or ``poetry`` to be used in
subsequent pipeline steps. It is important you pass the correct files:

- if your package uses python dependencies, pass the ``poetry.lock``
  and ``pyproject.toml`` files.
- if your repository builds a helm chart, pass the ``values.yaml``
  and ``Chart.yaml`` files

To make sure that a project gets tested against updated dependencies at
least once a day, create a pipeline schedule in GitLab that builds
the project with the ``skart-update`` job once every night.


.. _multi_repo_ci:

Connecting dependent repositories in CI
---------------------------------------

``skart`` is able to find development versions of dependencies from latest
builds of given branches. We can make use of this property and set up
a chain of builds of several dependent repositories so that each one of them
is built with its most recent dependencies and these most recent builds are
then used further in other, dependent repositories.

For example, we have the following dependency tree::

        ska-sdp-integration -> ska-sdp-proccontrol  -> ska-sdp-config
                            -> ska-sdp-helmdeploy   -> ska-sdp-config

In this example, ``ska-sdp-integration`` depends on two componentes,
both of which depend on ``ska-sdp-config``.

We can set up scheduled updates of all four of these repositories using
the ``deps-update`` make target (since that also updates with ``poetry``).
If we use this make target, it is OK if one or more of the dependent repositories
do not have a ``skart.toml`` file because they don't depend on any SKA packages.
In such cases, ``poetry`` will still run and update the ``lock`` file (if exists).
Set them up in the following order (times are purely for demonstration purposes, you
can shift them as needed, but the order needs to stay):

- start a build of ``ska-sdp-config`` at 2:00 am, running ``make deps-update``
  in the ``.pre`` stage.
- the pipeline using the same set up for ``ska-sdp-helmdeploy`` and
  ``ska-sdp-proccontrol`` needs to run about 30 minutes later, e.g. at 2:30 am.
  This will make sure the pipeline on ``ska-sdp-config`` already finished, and ``skart``
  on the other two repositories will use the artefact built as part of the 2:00 am-schedule,
  already with updated dependencies.
- finally, run the scheduled, ``skart``-based pipeline for ``ska-sdp-integration`` at
  3:00 am, to give time for ``ska-sdp-proccontrol`` and ``ska-sdp-helmdeploy`` pipeliens
  to finish, so that ``skart`` can update using these most recently built versions of the
  two components.

If for any reason the given pipelines take longer to finish than the delay
in setting up the schedules, ``skart`` will wait for any pipelines that are currently
running, to finish, before moving on with the update.

.. warning::

    Note however, that if any of the pipelines fail, ``skart`` will skip the build
    of the failed pipeline, and move onto the previous, successful one. This will
    result in using a dependency that is in fact NOT built with the latest code.