Merge request quality checks#

SKAO uses a series of automated checks to ensure Merge Requests follow the Software Quality Assurance processes, guidelines, and policies described in this Developer Portal. Marvin reports the results on the Merge Request page. Marvin reports the results of these checks in the main Merge Request page on GitLab. Look for this comment and respond to any issue arising.

A check is either a:

Failure (🚫): The Merge Request is violating the SKA guidelines and you must fix it by following the mitigation defined in the check
- Example
  - Branch name must start with a lowercase Jira Ticket ID - see Branching policy
Warning (⚠): The Merge Request is following anti patterns/non-advised guidelines/policies and it would be better if it is fixed by the mitigation defined in the check
- Example
  - Docker Compose commands found on the repository
Information (📖): Be aware of the information conveyed in this Merge Request quality check message
- Example
  - The merge request does not present documentation changes

Each check has a brief description that explains what it does and a mitigation/explanation (depending on check type) which gives detailed information about the check and how to fix it or explains its findings more. You can find a list of each check below.

Workflow#

When a new Merge Request is created, a webhook triggers the SKA MR Service to carry out the checks described below and Marvin the Paranoid Android (username: marvin-42) happily reports back to the Merge Request by adding a comment (probably the first comment). The comment includes a table (like the example below) with each check and associated information. Marvin also includes a random quote at the end of the table while staying in the spirit of the paranoid android (as he is from the Hitchhiker’s Guide to the Galaxy). These quotes are handwritten from actual quotes or AI assisted and they are meant to be fun and light-hearted to make the process more enjoyable and make the table more appealing to check.

For subsequent changes pushed to the Merge Request, Marvin updates the comment to reflect the latest status of the Merge Request.

Table with marvin feedback type (error/warning), description and mitigation strategy — Marvin’s Check Table.#

Note

Two services share the same persona. marvin-42 posts the Merge Request checks described on this page; marvin-42-service posts the artefact validations described in Artefact validations.

Besides the check table, Marvin also:

Publishes a GitLab external status check named Marvin on the Merge Request, so the check result can be required before merging.
Maintains a separate Dashboards and Logs comment holding links to the namespaces, dashboards and logs associated with the Merge Request’s pipelines.

Marvin approval#

After creating the table, Marvin checks whether any check in the failure category failed. If one did, Marvin does not approve the MR, and unapproves it if he had approved it before. If no check in the failure category failed, Marvin approves the MR.

Runtime configuration#

Not every check runs on every repository, so the table you see may be shorter than the list below. Which checks run is decided at runtime by, in order of precedence:

Project topics. A project carrying the external topic is skipped entirely.
Check filters. A filter file in the service repository names subgroups and projects that run only a subset of the checks, or that skip specific ones. Projects that do not use the BSD 3-Clause licence, for example, skip the licence check.
Feature toggles. Each check has its own feature toggle, managed through the GitLab Unleash integration, that enables or disables it globally.

If a check you expect is missing, or one you do not expect appears, raise it with the System Team Support Centre at the System Team service desk.

Checks#

Type	Description	Mitigation Strategy
Failure	Jira ticket ID	Branch name should start with a lowercase Jira ticket id (i.e. st-0000-id-checks). Please close this MR, rename your branch and create a new MR. Following commit messages violate Committing code: At commit: <commit-hash> At commit: <commit-hash> Please change the MR title to include a valid Jira ticket id in uppercase (i.e. ST-0000: id-checks). The Jira ticket ids used in your branch name, MR title or commits are different from each other. Please make sure this is intended, otherwise change them to match.
Failure	Missing Assignee	Please assign at least one person for the MR
Failure	Non-compliant License Information	All projects should use a BSD-3 licence as per the SKAO Licence Template. If you have a requirement to use a different licence please consult with the Architecture Team before adding the licence. Please update the license information according to Licensing a project. This check can only examine the default branch, so don’t worry if this message appears on a merge request that adds a compliant LICENSE file.
Failure or Warning	Outdated Helm Charts	The repository contains helm charts with outdated dependencies, please update the following: Chart: <chart_path> <chart-name>: using <version>, <available-version> available. <chart-name>: <version> is deprecated! Please use a version outside the range <version-range>.
Warning	Docker Compose Usage	Please remove docker-compose from following files. <file_location> on line <line_number> <file_location> on line <line_number>
Warning	Missing CODEOWNERS file	Please add a CODEOWNERS file to the root folder.
Warning	Missing CHANGELOG file	Please add a CHANGELOG.md file to the root folder.
Warning	Makefile submodule outdated	The .make submodule used in this project is outdated. Please consider running `make make` to update the reference.
Warning	Makefile Templates Usage	The repository is not using any of the CI/CD Makefile, please include the following templates: <makefile-include> (for <description>)
Warning	Pipeline Templates Usage	Please create a pipeline for this MR OR, if a pipeline exists but the CI/CD templates are missing: file: <template-include> # <description>
Warning	Poetry Usage	Please update the dependency management of this repository: Please set up poetry as a dependency manager Remove the requirements.txt usage in <file_location> Remove the setup.py usage in <file_location>
Warning	ReadTheDocs Integration	Please integrate this project with ReadtheDocs following the guidelines: Please set up docs/ folder for sphinx documentation build following the guidelines Please add this project as a subproject on Read the Docs following the guidelines Please import your project into Read the Docs
Warning	Missing Test Coverage	This Project is missing test coverage. Please have a look at the following page
Warning	Non-Complaint Project Slug Name	Project Slug should start with ska-. To change the slug go into: Settings->Advanced->Change Path If you don’t have access to this settings, please contact the system team at #team-system-support slack channel
Warning	Repository Structure is not following standardised Project Structure	Following rules failed for the repository structure: Python: There should be `pyproject.toml` file in the root folder Python: Python files should be under a python module starting with ska_(preferably project_slug) in the `src/` folder Helm: There should be at least one chart in the `charts/` folder Helm: Charts should start with <project_slug> prefix … See repository structure guidelines for all of the rules
Information	Documentation Changes	This MR doesn’t introduce any documentation changes. Please consider updating documentation to reflect your changes.

Jira ticket ID#

This check verifies the Jira ticket ID in the branch name, every commit message on the Merge Request, and the Merge Request title. Then it verifies that the IDs found agree with each other. Having the Jira ticket ID makes every branch, commit and Merge Request identifiable with its ticket (through the GitLab/Jira integration), which is a key part of the Software Quality Assurance programme and how developers track changes made on a ticket.

The branch name must start with a lowercase Jira ticket ID, as described in Trunk-based development. Commit messages must carry an uppercase ticket ID, as listed at Committing code; merge and revert commits are exempt. The Merge Request title must include an uppercase ticket ID, optionally behind a Draft: prefix. In each case the ID must resolve to a real Jira issue, not merely match the pattern. Marvin’s own automation branches, commits and titles (prefixed marvin- or Marvin:) are exempt from all three.

The level of this check is a failure, except in one case: when the branch, commits and title each carry a valid ID but the IDs differ from one another, the check is reported as a warning instead, because a deliberate mismatch is sometimes intended.

Missing assignee in MR#

This check warns users that no one is assigned to the Merge Request. The level of this check is failure. To avoid it, add at least one assignee to the MR.

Non-compliant license information#

This check warns users if the license in their project is not compatible with an SKA approved license so that the quality of the software is improved and compliance is ensured with SKA standards and obligations to GitLab. All projects must use a BSD 3-Clause licence; if you have a requirement to use a different one, consult the Architecture Team first. The level of this check is a failure. The check only examines the default branch, so it can still report on a Merge Request that adds a compliant LICENSE file. Projects in the ‘External’ group are skipped, and a small set of projects with a documented need for a different licence are exempt through the check filter file.

Outdated Helm charts#

This check inspects the chart dependencies declared under charts/ and compares each one against the versions published in the SKAO Helm chart repository. It reports a warning when a dependency is merely behind the latest published version, and a failure when a dependency uses a version that has been explicitly deprecated. Version strings that cannot be parsed are also reported. To avoid it, update the dependency versions listed in the mitigation.

Docker Compose usage#

This check warns users that they are using Docker Compose in their project. This will make it easier to remove Docker Compose from the projects, as it must no longer be used (it creates issues with the underlying networks). To avoid this warning, remove Docker Compose from the project. The details of the files involved can be seen in the warning message under the Mitigation Strategy column along with the line numbers where Docker Compose is found.

Missing CODEOWNERS file#

This check warns users not to merge their Merge Request without a CODEOWNERS file present in the root folder of the repository. This file specifies who owns the code and can be used to automatically set up who can approve Merge Requests.

Missing CHANGELOG file#

This check warns users that the repository has no CHANGELOG.md file in its root folder. The changelog is what the automated release process uses to describe each release, so a missing one leaves releases undocumented. The level of this check is a warning.

Makefile submodule outdated#

This check warns users that the .make submodule referenced by the repository is behind the current release of the pipeline machinery, which means the project is missing fixes and targets available upstream. The level of this check is a warning. To avoid it, run make make to update the submodule reference and commit the result.

Makefile templates usage#

This check warns users that the repository’s Makefile does not include the pipeline machinery templates that match the kinds of artefact it builds — for example .make/oci.mk for OCI images or .make/python.mk for Python packages. The level of this check is a warning. To avoid it, include the templates listed in the mitigation, or include .make/*.mk to cover all of them. If there is a reason the project cannot use them, contact the System Team on the #team-system-support Slack channel.

Pipeline templates usage#

This check warns users that the repository either has no pipeline at all, or has one that does not include the shared CI/CD job templates appropriate to its contents. The level of this check is a warning, and to avoid it 2 steps may be needed. The first one is to create a pipeline (i.e. add .gitlab-ci.yml) if there is not one created yet. The second one can only be done after the first one, and it consists of including the templates that are listed on the mitigation strategy column. If there is a reason the project cannot use them, contact the System Team on the #team-system-support Slack channel.

Poetry usage#

This check applies to Python repositories and warns users that the project is not managing its dependencies with Poetry, or that it still carries files that Poetry replaces — requirements.txt or setup.py. The level of this check is a warning. To avoid it, add a pyproject.toml to the root folder and remove the files listed in the mitigation.

ReadTheDocs integration#

This check warns users that this project does not follow the guidelines for successful Read The Docs Integration. The level of this check is a warning. To avoid it, three checks must pass. Set up a docs/ folder for the sphinx documentation build, add the project as a subproject on Read the Docs, and finally set up the webhooks that import the project into Read the Docs. Documenting a project walks through all three steps. See Integration into the Developer Portal for how a project’s documentation hangs off the Developer Portal, and add the project as a sub-project for the sub-project step in particular.

Missing test coverage#

This check warns users if test coverage is missing, by verifying if the file code-coverage.xml exists in the .post job (create-ci-metrics). To avoid it, make sure your tests are exporting a build/reports/code-coverage.xml file, or simply use the make submodule targets. Unlike the other checks on this page, this one runs on pipeline events rather than Merge Request events, so it only reports once the pipeline has finished.

Non-compliant project slug name#

This check warns users if the project slug does not have a compliant name. For the slug to be compliant, it must start with ska-. The level of this check is warning, to fix it, go to: Settings → Advanced → Change Path and change the slug to a compliant name. If you do not have access, contact the System Team on the #team-system-support Slack channel.

Repository structure compliance#

This check warns users if the project structure and content is not following the SKAO standardised Project Structure. The level of this check is warning. To fix it, comply with all of the rules listed in the Mitigation Strategy column. If you think there is an error or bug, contact the System Team Support Centre at the System Team service desk.

Documentation changes#

This check warns users that this MR doesn’t introduce any documentation changes. A change counts as documentation if it touches docs/src, docs/source, a README or a CHANGELOG. This check provides information only. Action is not always needed, as some MRs may not require a documentation update; it is just a gentle warning that a documentation update might be missing.