SKA CI/CD Automation Services MR Checks

This plugin is used to check MR quality and provide feedback on the MR window by making comments and updating them.

It uses FastAPI to create webhook that can be set to listen for the GitLab Projects. The following environment variables must be present, the token should have API access to the project:

PRIVATE_TOKEN=...
REQUESTER=...
JIRA_URL=...
JIRA_USERNAME=...
JIRA_PASSWORD=...
GITLAB_TOKEN=...
GITLAB_HEADER=...
UNLEASH_API_URL=...
UNLEASH_INSTANCE_ID=...
RTD_TOKEN=...

Checks

Each check should have:

  • Feature Toggle Name: name of the check for runtime configuration

  • Result Type: If the check is not successful, whether it should be marked as FAILURE, WARNING, or INFO

  • Description: Brief description about what the check is about

  • Mitigation Strategy: How to take corrective action to fix the broken check

Currently the plugin checks the MR are:

Type Description Mitigation
Warning Missing Test Coverage This Project is missing test coverage
Please have a look at the following [page](https://developer.skatelescope.org/en/latest/tools/ci-cd/continuous-integration.html?highlight=coverage#automated-collection-of-ci-health-metrics-as-part-of-the-ci-pipeline)
Failure Missing Assignee Please assign at least one person for the MR
Failure Source Branch Delete Setting Please check "Delete source branch when merge request is accepted.
Failure Missing Jira Ticket ID in MR Title Please uncheck Squash commits when merge request is accepted.
Warning Docker-Compose Usage Found Please remove docker-compose from following files:
  • At file: `file_location` on line `line_number`
  • At file: `file_location` on line `line_number`
Warning Missing CODEOWNERS file Please add a CODEOWNERS file to the root folder.
Failure Missing Jira Ticket ID in Branch Name Branch Name should start with a Jira ticket
Warning Missing Jira Ticket ID in commits Following commit messages violate the formatting standards:
  • At commit: `commit_hash`
  • At commit: `commit_hash`
Warning Non-compliant License Information Please update the license information according to developer portal guidelines for your project
Information Documentation Changes This MR doesn't introduce any documentation changes. Please consider updating documentation to reflect your changes
Failure ReadTheDocs Integration Please integrate this project with ReadtheDocs following the guidelines:
  • Please set up docs/ folder for sphinx documentation build following the guidelines
  • Please add this project as a subproject on Read the Docs following the guidelines
  • Please import your project into Read the Docs
Failure Wrong Merge Request Settings Reconfigure Merge Request Settings according to the guidelines:
  • MR Settings Checks:
  • You should assign one or more people as reviewer(s)
  • Automatically resolve mr diff discussions should be checked
  • Override approvers and approvals per MR should be checked
  • Remove all approvals when new commits are pushed should be checked
  • Prevent approval of MR by the author should be checked
  • There should be at least 1 approval required
  • Please check Delete source branch when merge request is accepted.
  • Please uncheck Squash commits when merge request is accepted.
Project Settings Checks (You may need Maintainer rights to change these):
  • Pipelines must succeed should be checked
  • Enable Delete source branch option by default should be checked
  • Show link to create/view MR when pushing from the command line should be checked
Failure Could not find needed pipeline jobs

Please create a pipeline on this Merge Request

OR

Please add the following jobs:

Automatic Fixing of Wrong Merge Request Settings

Marvin will attempt to automatically check the delete source branch and uncheck the squash commits on merge settings. Next to each of the other Wrong Merger Request Settings messages is a Fix link, which will trigger Marvin to attempt to fix that setting after the user is authenticated. Only users that are assigned to the merge request can trigger this automatic setting fix feature.

Marvin MR Approval

Marvin after creating the table will verify if there is any checks under the failure category failed, if so Marvin does not approve the MR, and in the case that that MR was already approved before by him he unapproves it. If none of the checks under the failure category failed Marvin will approve the MR.

Runtime Configuration

This service is using feature toggles to determine which checks to enable/disable at the runtime. It uses Unleash integration provided by GitLab to achieve this.

For the project level configuration, a project could be disabled using Project Tags/Topics. The service uses a blocklist to determine whether it should run the checks as well.

Precedence of configuration

  • Project Level Configuration with Tags/Topics

  • Feature Toggle Strategies

How to Add a New Check

Each new check must use the abstract base class, Check, to ensure to define its type, description, mitigation strategy and check action, which performs the actual checking on the MR and returns a boolean indicating the result.

Base Class:

class Check(ABC):

    feature_toggle: str = NotImplemented

    @abstractmethod
    async def check(self, mr_event, proj_id, mr_id) -> bool:
        pass

    @abstractmethod
    async def type(self) -> MessageType:
        pass

    @abstractmethod
    async def description(self) -> str:
        pass

    @abstractmethod
    async def mitigation_strategy(self) -> str:
        pass

Example Check:

class CheckAssigneesComment(Check):
    feature_toggle = "check-assignees-comment"

    def __init__(self, api: GitLabApi, logger_name):
        self.api = api
        self.logger = logging.getLogger(logger_name)

    async def check(self, mr_event, proj_id, mr_id):
        mr = await self.api.get_merge_request_info(proj_id, mr_id)
        self.logger.debug("Retrieved MR: %s", mr)
        return len(mr["assignees"]) > 0

    async def type(self) -> MessageType:
        return MessageType.FAILURE

    async def description(self) -> str:
        return "Missing Assignee"

    async def mitigation_strategy(self) -> str:
        return "Please assign at least one person for the MR"

Then the necessary tests for the added checks should be added in tests folder. These tests should get picked up by the main frameworks testing.

Finally, each check should be initialised and called in the mrevents file to be included into the list of checks that are performed for the MRs.