CI/CD Reference#

Quick reference for pipeline stages, variables, runner tags, and metrics at SKAO.

Key URLs#

Resource

URL

SKA Group

gitlab.com/ska-telescope

Templates Repository

ska-telescope/templates-repository

Python Skeleton

ska-telescope/templates/ska-python-skeleton

Central Artefact Repository

See Central Artefact Repository

SKA CICD Makefile

ska-telescope/sdi/ska-cicd-makefile

Pipeline Stages#

All projects must include these stages:

Stage

Description

Output

Build

Compiles software into distributable units

.deb, .whl, .jar, or Docker image

Linting

Analyzes code statically

Quality analysis results in JUnit format

Test

Runs tests on built software

Test results in JUnit format, coverage metrics

Publish

Uploads artefacts to SKA software repository

Packages in CAR

Scan

Scans containers for security vulnerabilities

Security scan results in JUnit format

Pages

Publishes results to GitLab Pages

HTML with pipeline test results

Documentation

Generates project documentation via ReadTheDocs

HTML documentation

Test Types#

In practice, SKAO distinguishes between unit tests — from which CI metrics are collected — and post-deployment tests, which do not feed into CI metrics.

Type

Description

Unit tests

Test smallest units in milliseconds. Run locally without filesystem, network, or database.

Component tests

Test individual components in isolation.

Integration tests

Test interactions between components. Verify Docker images work as expected.

Deployment tests

Verify software deploys correctly and behaves as expected.

Configuration tests

Test multiple combinations of software and hardware.

System tests

Test complete solution with integrated hardware and software.

Runner Tags#

Use these variables in your .gitlab-ci.yml instead of hardcoding tags:

Variable

Default Value

Use Case

SKA_DEFAULT_RUNNER

ska-default

Standard pipeline jobs

SKA_K8S_RUNNER

ska-k8s

Kubernetes deployment/integration testing

SKA_GPU_RUNNER

ska-gpu-a100

GPU workloads

GPU tags:

  • ska-gpu-a100 — NVIDIA A100 GPUs

  • Additional GPU tags per cluster

Clusters:

Cluster

Description

techops

Main cluster for CI/CD. Limited GPU support, primarily for building artefacts.

dp

Data Processing ART cluster. More GPUs for running actual workloads.

Metrics Report Requirements#

Reports must follow these requirements for automated collection:

  1. Files must not be part of the repository — create them in test or linting stages

  2. Unit tests report: JUnit XML at ./build/reports/unit-tests.xml

  3. Linting report: JUnit XML at ./build/reports/linting.xml