SKA PST

Documentation Status

The ska-pst repository integrates the software subcomponents of the Square Kilometre Array (SKA) Pulsar Timing (PST) instrument and provides a Helm chart that facilitates the deployment of PST. The documentation for this project can be found at SKA developer portal

Download the source code

First, clone the repository and all of its submodules

git clone --recursive git@gitlab.com:ska-telescope/pst/ska-pst.git

Next, change to the newly cloned directory and create the build/ sub-directory

cd ska-pst
mkdir build

Build Instructions

The following instructions describe how to build the software using the same docker image that is used in the ska-pst gitlab pipelines. To build directly on your host machine, see the Developer Setup instructions below.

Launch the builder image as a container. Note the current working directory will be mounted into the container as /mnt/ska-pst.

make local-dev-env

After the builder container is launched, you can run any of the following C++ Build commands. To run either the Documentation Build or Python Linting commands, you’ll also need to install and run poetry.

pip install poetry
poetry self add poetry-plugin-shell
poetry shell
poetry install

DSPSR

If developing concurrently in DSPSR it is useful for a developer to have to avoid making changes in DSPSR, force a build of the ska-pst-dspsr Docker image in ska-pst-dsp-tools. This is why the DSPSR repository has been added as Git submodule of this project.

A developer can use the dspsr submodule as their working copy of DSPSR. However, this is currently set as a read-only checkout of the code but a developer can locally add a read-write upstream view of this and work on that and push code changes to the upstream

cd resources/dspsr
git remote add upstream ssh://<username>@git.code.sf.net/p/dspsr/code
git checkout -b <branch-name>

After creating the branch and making changes they can the push the changes to the upstream not origin branch.

git push --set-upstream upstream <branch-name>

NOTE that during a merge request (MR) on SKA PST that the DSPSR submodule should be pointing the latest master that includes these changes (i.e. a MR in DSPSR has been accepted) and merged. After that MR has been approved but you need to do the manual merging onto master do the following:

git checkout master
git merge <branch-name>
git push upstream master
git branch -d <branch-name>
git push upstream -d <branch-name>
git fetch origin -v
git pull --ff origin master

The above commands do the following:

  • sets current branch as master

  • does the merge of the branch onto master

  • pushes merged master branch to the editable upstream remote

  • deletes the branch locally

  • deletes the branch off the remote upstream

  • updates the git information for origin

  • updates the master branch from origin which is effectively the same as the upstream remote. This operation should do nothing if the git fetch origin -v worked

If a new version of DSPSR Tools is built then update the following:

  • In Makefile - ensure DEV_IMAGE and DEV_TAG is pointing to the correct location, also check your own PrivateRules.mak

  • In .gitlab-ci.yml - update SKA_PST_DSP_TOOLS_IMAGE in the variables to point to the correct tag of the DSPSR builder image (e.g. ska-pst-dspsr-builder).

Debug C++ Build

The debug build will use the Cmake build arguments -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_FLAGS="-coverage" -DCMAKE_EXE_LINKER_FLAGS="-coverage". For debug purposes, the #define DEBUG will be defined for the software to enable any debug features.

make -j local-cpp-build-debug

Release C++ Build

The release build will use the Cmake build arguments -DCMAKE_BUILD_TYPE=Release which ensures #define DEBUG is not defined. This build should be used for all deployments.

make -j local-cpp-build-release

Linting C++ Build

This build target compiles the library and applications with the flags required for linting and static analysis, and behaves like the lint_cpp_clang_tidy jobs in the Gitlab pipeline.

make local-cpp-ci-simulation-lint

During the compilation, the build generates compile_commands.json file which is used in the linting and static analysis tools: clang-tidy and cppcheck.

Documentation Build

API documentation for the library is generated with Doxygen, which is then converted into ReadTheDocs format by Sphinx, Breathe and Exhale. The documentation may be built in the dev-env via

apt update
apt install pandoc
pip install poetry
poetry self add poetry-plugin-shell
poetry shell
poetry install --no-root --only docs
make docs-build html

Python Linting Build

This project requires that the code is well formatted and linted by pylint and mypy.

Your code can be formatted by running:

make python-format

While the code can be linted by running:

make python-lint

To ensure that formatting happens before linting, add the following to PrivateRules.mak

python-pre-lint: python-format

.PHONY: python-pre-lint

Developer Setup

The tests of this project use PyTango via the ska-tango-base project.

To make sure your development environment is ready, follow the Installation instructions of the ska-tango-examples project (this is specific for Ubuntu but you should be able to work it out for other environments).

At the very least have Docker and install Minikube - (see - SKA Deploy Minikube).

Required Packages

The following packages are required for local development of this project. If the developer is not using Ubuntu or Window Subsystem for Linux (WSL) then they will need to find an equivalent package.

sudo apt-get update
sudo apt-get install doxygen graphviz

Poetry Setup

No matter what environment that you use, you will need to make sure that Poetry is installed and that you have the Poetry shell running.

Install Poetry based on Poetry Docs. Ensure that you’re using at least 2.1.3, as the pyproject.toml and poetry.lock files have been migrated to the Poetry 2.1.3.

If the first time of installing poetry or upgrading to Poetry 2.x you will want to install the poetry-plugin-shell plugin

poetry self add poetry-plugin-shell

To activate the poetry environment then run in the same directory:

poetry self add poetry-plugin-shell
poetry shell

After having Poetry installed, run the following command to be able to install the project. This will create a virtual env for you before starting.

poetry install

If this is successful you should be able to use your favourite editor/IDE to develop in this project.

(For VS Code, you can then set your Python Interpreter to the path of this virtual env.)

Ensure Linting Before Commit

It is highly recommended that linting is performed before committing your code. This project has a pre-commit hook that can be enabled. SKA Make machinery provides the following command that can be used by developers to enable the lint check pre-commit hook.

make dev-git-hooks

After this has been applied, git commit commands will run the pre-commit hook. If you want to avoid doing that for some work in progress (WIP) then run the following command instead

git commit --no-verify <other params>

Editor Configuration

This project has an .editorconfig file that can be used with IDEs/editors that support EditorConfig. Both VS Code and Vim have plugins for this, please check your favourite editor for use of the plugin.

For those not familiar with EditorConfig, it uses a simple configuration file that instructs your editor to do some basic formatting, like tabs as 4 spaces for Python or leaving tabs as tabs for Makefiles, or even trimming trailing whitespace of lines.

Layout of the Software

SKA PST is composed up of six subcomponents. The code for each of the subcomponents is distributed in the python/src/ska_pst, src/ska_pst, and src/apps sub-directories; e.g.

  • python/src/ska_pst/lmc

  • src/ska_pst/dsk/disk

License

See the LICENSE file for details.