GitLab CI Configuration
At present all CI configuration is in one file, gitlab/build.yml.
Importing CI Config to Your Project
As mentioned in the README, you’ll need to add this common project as a git submodule, and also include the CI configuration in your .gitlab-ci.yml file.
Here’s a complete example taken from ska-low-cbf-fw-pst (as of 29 June 2022)
variables:
GIT_SUBMODULE_STRATEGY: recursive
XPFM: "/opt/xilinx/platforms/xilinx_u55c_gen3x16_xdma_2_202110_1/xilinx_u55c_gen3x16_xdma_2_202110_1.xpfm"
TARGET_ALVEO: "u55"
include:
# Alveo FPGA build common CI configuration
# Note: GitLab includes always grab the latest.
# This may not match the copy in our 'common' submodule!
- project: 'ska-telescope/low-cbf/ska-low-cbf-fw-common'
file: 'gitlab/build.yml'
# SKA standard includes
# RAW artefact handling
- project: 'ska-telescope/templates-repository'
file: 'gitlab-ci/includes/raw.gitlab-ci.yml'
# Release management
- project: 'ska-telescope/templates-repository'
file: 'gitlab-ci/includes/release.gitlab-ci.yml'
# Docs pages
- project: 'ska-telescope/templates-repository'
file: 'gitlab-ci/includes/docs.gitlab-ci.yml'
# .post step finalisers eg: badges
- project: 'ska-telescope/templates-repository'
file: 'gitlab-ci/includes/finaliser.gitlab-ci.yml'
raw-build:
# note: this is modifying a job imported from templates-repository!
needs:
- job: v++
optional: true # v++ job doesn't always exist in pipeline
variables:
BUILD_STRING: "CI CAR Package. Pipeline ${CI_PIPELINE_ID} created at ${CI_PIPELINE_CREATED_AT} on branch ${CI_COMMIT_BRANCH}. Commit ${CI_COMMIT_SHORT_SHA}."
before_script:
# we check if the build directory exists because v++ doesn't always run
# CI=false is a hack to stop the package_firmware script from uploading anything.
- if [ -d build ]; then CI=false common/scripts/package_firmware.sh; fi;
Testing a Development Branch
In your downstream project, edit .gitlab-ci.yml using the ref
keyword to
include your ska-low-cbf-fw-common development branch
include:
- project: 'ska-telescope/low-cbf/ska-low-cbf-fw-common'
file: 'gitlab/build.yml'
ref: 'perentie-1234-fw-common-branch-name'
If you’re modifying shell scripts, you’ll also need to modify the common submodule to point to the branch. This sequence of commands may help…
cd common
git checkout perentie-1234-fw-common-branch-name
cd ..
git add common
git commit
Variables
A variable defined in build.yml is assumed to be mandatory unless it contains the word “EXTRA”. A sensible default value should be provided if possible, but mandatory variables may be left empty in build.yml, leaving downstream projects to provide their value in their .gitlab-ci.yml file.
BASEDIR Note
The BASEDIR
variable is used to provide the absolute path to the base
directory where the build is starting from. As this is not known at CI pipeline
instantiation time, we have to use some tricks to evaluate it later, when the
jobs are running.
So you will see $${BASEDIR}
in the YAML, escaping the $ so GitLab does
not interpret it. The variable BASEDIR
is set to the repositry root in
util.sh. Then, at the time of use, envsubst
is used to substitue the
current value in.
(Perhaps all this complexity could go away if the TCL scripts / tools were able to use relative paths instead??)
Runner Configuration
Due to licensing restrictions, pipeline access is locked down to members of the SKA Perentie Team (aka CSIRO S&A Employees).
See Alveo Gitlab Runner on SKA Confluence for installation instructions.
Pipeline Stages
Stages are copied from the standard SKA pipeline for compatibility.
Pipeline Jobs
As the build process takes a long time and we have limited servers to run on, the job steps are dynamic and the full build is not always run.
The following build processes are supported:
Building the FPGA design will be triggered via:
Commit to a branch with keyword build in the commit message.
Merge Request. This is to test timing on the work before merging.
Any commit on the main branch. This is to make sure main is always in a working state.
Tag, which triggers a pipeline including publication to the Central Artefact Repository (CAR).
Commit documentation, release notes or build information. Due to restrictions of committing to main, there is a commit message key word “docs”, that will not trigger FPGA build jobs but just run up to FPGA lint on a branch. Note: Jobs included from other sources will still run.
The magic word “docs” needs to be used in the commit first line and in the branch name due to how MR to main adopts the branch name.
Simulation jobs:
Manually triggered long and short simulation jobs are available.
The short sim will run on a merge request or tag, long will not be required.
See the rules
in build.yml for exact details.