Motivation

The aim of skart is to enable continuous integration practices – in the interpretation taken for instance by Martin Fowler:

  • Put everything in a version controlled mainline

    […] Nobody should ever ask “where is the foo-whiffle file?” Everything should be in the repository. […]

    The SKA development implements this principle – however we have to see the collective of all GitLab repositories hosted under https://gitlab.com/ska-telescope as “the version controlled mainline” for the purpose of this discussion. This fragmentation has a number of ramifications, as we will see below.

  • Automate the Build

    […] anyone should be able to bring in a clean machine, check the sources out of the repository, issue a single command, and have a running system on their own environment. […]

    If we see “the repository” as the collective of all GitLab repositories here, this is in fact an almost unfeasibly laborious task: We would have to manually check out and install all repositories one by one.

    This especially becomes relevant if we consider what automated builds should enable – generating consistent releases. In a situation where the code is spread across multiple repositories with “locked” dependencies between each other, making a consistent release of the collective “mainline” becomes a lengthy process of tagging, generating packages, updating dependencies etcetera.

    skart aims to help with this by automating recursive walks across repositories and finding appropriate packages of dependencies. This comes close to the spirit of the principle as long as we are willing to believe that GitLab runners generate correct packages.

  • Everyone Commits To the Mainline Every Day

    […] Integration is primarily about communication. Integration allows developers to tell other developers about the changes they have made. Frequent communication allows people to know quickly as changes develop. […]

    On one hand, it can be argued that splitting repositories does not inherently cause a problem with this principle, as clearly commits to any individual repository are by our definition a commit to “mainline”. However, this misses the actual intent of the principle: to ensure that all developers are testing against a shared source of truth.

    For this to be effective, we have to ensure that developers are constantly updating their dependencies to match what others have developed - and especially not “just” what they have released! This is again what skart aims to help with, as it allows developers (as well as the CI system) to easily pull in the most current packages of all dependencies.

  • Every Push to Mainline Should Trigger a Build

    […] We thus need to ensure that every commit is verified in a reference environment. The usual way to do this is with a Continuous Integration Service (CI Service) that monitors the mainline. […]

    This is a similar situation to the previous principle, but now from the perspective of GitLab CI: While developers might be willing to manually track down development versions of each other’s packages, this is clearly not an option for automated builds. As a result, GitLab CI often builds against dependencies that are older than what would be considered “mainline.”

    This is again where skart can help: By automatically discovering the most newly-built packages from other repositories we can guarantee that we always build a consistent overall codebase.