Automated Release Process

The components of the SKA SDP dependency structure are rather intricate. Releases are made manually, which are a tedious task and results in a lot of time overhead for those going through the process. For instance, several SDP control system components, processing scripts, and other packages directly depend on the Configuration Library, which is located at the bottom of the tree as shown in the diagram below. After it is updated and released, all dependent projects must also be updated to utilise the latest release and then they themselves need to be released. Consequently, the idea emerged of developing a process that would automatically update dependencies, test the components using the new dependencies, and then make the releases.

../_images/sdp_release.svg

SDP Release Diagram which shows the dependencies between components, processing scripts and packages

Update and Release Targets

In the Standard Make File repository, the following make targets are available, which can be used for updating dependencies and making related releases:

  • deps-update: which updates the dependencies (development versions) using SKA Inter-repository Tooling (SKART) and poetry lock.

  • deps-update-release: this target updates the dependencies using skart and poetry. It is done in the relase mode, which means it updates the packages listed in the skart.toml file with the given release specifications and then updates poetry lock.

  • deps-patch-release: after updating the dependencies with released versions (see deps-update-release), it bumps the patch version of the current repository (if there are any changes to be released), creates a git tag and pushes the changes to master/main.

Recursive Behaviour

Recursive behaviour operates by updating dependencies, testing the components with the new dependencies, then making the releases automatically. This function is implemented in SKA Inter-repository Tooling (SKART).

In order to invoke this behaviour, you first need to install skart

$ pip3 install \
--index-url https://artefact.skao.int/repository/pypi-all/simple \
skart

Then invoke the command

$ skart invoke-recursive -- make deps-patch-release

The steps above command will do are as follows:

  • Starts by searching for the skart.toml file in the current repository.

  • Once discovered, it will go through all the dependencies and look for its own skart.toml file until none are found.

  • It will then clone the each repositories from the base of the dependency tree temporarily and then executes the deps-patch-release target. It will only move to the next sub-component once it has released and pushed the current one to Central Artefact Repository (CAR).

Note - Please make sure not to run the above command with -- make deps-patch-release target for experiments. Use something like echo Hello as an example.

More Details and instructions about the recursive behaviour and the command can be found here