Integration Testing CI Pipeline

Integration tests are run in the Low PSI, via GitLab job psi-low-test.

These tests are performed at the Tango control system level, using multiple Low CBF sub-components.

Version Specification

The CI tests are designed to be able to test a particular version of any sub component(s). Versions can be requested by upstream pipelines using these environment variables:

CNIC_FW_VERSION, CNIC_FW_SOURCE: CNIC Firmware
CORR_FW_VERSION, CORR_FW_SOURCE: Correlator Firmware
PST_FW_VERSION, PST_FW_SOURCE: PST Firmware
SKA_LOW_CBF_VERSION: Low CBF Monitoring & Control
SKA_LOW_CBF_PROC_VERSION: Processor Tango Device
SKA_LOW_CBF_TANGO_CNIC_VERSION: CNIC Tango Device

By default, these are sourced from the SKAO Central Artefact Repository. If a software version contains dev, the corresponding Helm chart & container image will be sourced from GitLab instead (using a GitLab project ID specified in Makefile).

Firmware uses a manually-specified source - either “gitlab” or “nexus” (CAR).

CNIC Tests

test_cnic_p4.py uses a CNIC in duplex mode & the shared PSI P4 switch to verify CNIC operation.

Processor Tests

test_processor.py shows an example of testing a Low CBF signal processing FPGA firmware image.

The example test function is test_correlator, which performs multiple Correlator tests using a requested firmware version.

The generic test routine (configure control system, transmit from CNIC, capture SUT output) is performed by the cnic_processor function.

The cnic_processor routine makes use of fixtures:

ready_subarray, which returns a Subarray device proxy that is, you guessed it, ready to configure.
cnic_and_processor, which returns a CNIC device proxy and a Processor device proxy. It configures network routing (using the Low PSI shared Connector device) between the CNIC & Processor. These routes are also cleaned after use.

A data class ProcessorTestConfig is used to collate the various test parameters.

FPGA Register Logging

If environment variable FPGA_LOG_REGISTERS is set to either 1 or true, the cnic_processor function will ask the Processor to log register activity to /test-data/<short commit hash> <timestamp> <test name>.log.

Manually Running the CI Pipeline

When developing a new test, you might want to avoid the round-trip delays of pushing to GitLab and waiting for the whole pipeline to run. Instead, you can use the run-psi-test.sh script to replicate the Low PSI test portion of the CI pipeline from the comfort of your own laptop. It accepts the same environment variables, and tries to make sensible guesses about what you’d like to do (e.g. using the latest integration container from your branch, falling back to the main branch if your branch doesn’t have one).

Extra Environment Variables for Manual Use

DEBUG_DIRTY: inherited from SKA Makefiles, this lets you run the tests when you have local uncommitted changes. Beware that if you have changed code in src (as opposed to tests), you need to push this to GitLab to get a new package built! If you know you have only changed tests, supply DEBUG_DIRTY=1 to run the script.
EXTRA_PYTEST_ARGS: as you might guess from the name, adds extra arguments to pytest. For example if you want to run ONLY your new test you can use EXTRA_PYTEST_ARGS=-k my_new_test.