Installation and Running of SKA SDP Benchmark tests

Installation instructions

ReFrame is developed in Python and hence, Python 3.6+ and several python modules are needed to run the tests. We strongly recommend to use conda environment to create virtual environments. Installation instructions for Spack is provided separately here in Installation section.

  1. Clone this repository:

git clone https://gitlab.com/ska-telescope/sdp/ska-sdp-benchmark-tests.git

  1. Install conda (if not already installed):

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

More instructions on installing conda can be found here

  1. Create and activate ska-sdp-benchmark-tests environment. The conda environment files are placed in folder share/conda/ in the root of the repository. Currently only x86 and ppc64le architectures are supported. For instance, if the system under test has x86 architecture, we can create conda environment using:

cd ska-sdp-benchmark-tests
conda env create -f share/conda/environment-x86.yml
conda activate ska-sdp-benchmark-tests

  1. Install all dependencies (like ReFrame) using the share/setup-env.sh script:

source /path/to/ska-sdp-benchmark-tests/share/setup-env.sh

This script installs the correct ReFrame version, wraps the reframe binary around a shell function and modifies the PYTHONPATH appropriately to get all modules on the path. Once this environment is set, we can invoke reframe from anywhere and run tests by providing an absolute path to test files. It also fetches and installs the SDP Perfmon toolkit, compiles it and installs it into the current conda environment. If the perfmon is installed, this script requires a working compiler toolchain and CMake installation. If you don’t want to install the perfmon, set "INSTALL_PERFMON=no" before sourcing the script.

Note

The installation of SDP Perfmon needs CMake >= 3.17.5.

Running SKA SDP Benchmark tests

As stated in the Installation instructions, once we set up environment for SKA SDP Benchmark tests using the script setup-env.sh, we can invoke reframe from anywhere in the system. All the documentation provided in the following subsections assume that this environment is not set up and hence all tests are invoked from repository root. For example, if a test mytest.py is located at an absolute path /path/to/mytest.py, we can execute that test using either:

cd ska-sdp-benchmark-tests
conda activate ska-sdp-benchmark-tests
source share/setup-env.sh
reframe -c /path/to/mytest.py -r