SKA-PST-SMRB

This project provides the C++ library and applications for the Shared Memory Ring Buffer component of the Pulsar Timing instrument for SKA Mid and SKA Low.

Documentation

Documentation Status

The documentation for this project, including the package description, Architecture description and the API modules can be found at SKA developer portal: https://developer.skao.int/projects/ska-pst-smrb/en/latest/

Build Instructions

Firstly clone this repo and submodules to your local file system

git clone --recursive git@gitlab.com:ska-telescope/pst/ska-pst-smrb.git

To simulate the build that will be performed in the Gitlab CI, a C++ builder image that has been extended from the ska-cicd-cpp-build-base C++ building image is used. This image includes the required OS pacakge and other custom software dependencies. The current version of this image is defined in .gitlab-ci.yaml.

docker pull artefact.skao.int/ska-pst-common-builder:0.10.5

Now change directory to ska-pst-smrb and launch this builder image as a container. Note the current working directory will be mounted into the container as /mnt/ska-pst-smrb.

make local-dev-env

The library and applications can be built using the standardise makefile templates provided by CICD infrastructure.

Debug Build

The debug build will use the Cmake build arguments -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_FLAGS="-coverage" -DCMAKE_EXE_LINKER_FLAGS="-coverage". For debug purposes, the #define DEBUG will be defined for the software to enable any debug features.

make build_debug

Release Build

The release build will use the Cmake build arguments -DCMAKE_BUILD_TYPE=Release which ensures #define DEBUG is not defined. This build should be used for all deployments.

make build_release

Linting Build

This build target compiles the library and applications with the flags required for linting and static analysis.

make local-cpp-ci-simulation-lint

During the compilation, the build generates compile_commands.json file which is used in the linting and static analysis tools: clang-tidy, cppcheck and IWYU.

Documentation Build

API documentation for the library is genereated with Doxygen, which is then converted into ReadTheDocs format by Sphinx, Breathe and Exhale. The documentation is built via

make docs-build html

Building Docker Image

To build an OCI Docker image for testing the applications:

make OCI_IMAGE_BUILD_CONTEXT=${PWD} oci-build-all

Tests

The unit tests for the library can be executed via the make target:

make local-cpp-ci-simulation-test

This will compile the software using the Debug Build and then execute test and test_memcheck make targets

Gitlab CI Simulation

To simulate the end to end build and test related jobs performed the gitlab runner

make local-cpp-ci-simulation

To simulate just the linting part of the simulation use:

make local-cpp-ci-simulation-lint

To simulate just testing use:

make local-cpp-ci-simulation-test

To generate code coverage reports, use:

make local-cpp-ci-simulation-coverage

Local Execution

After having built a local image, the runtime image can be used to run any of the executables that are within the container. Any files that need to be accessed from local/host will be mounted at /mnt/ska-pst-smrb

To run the info:

make local-docker-run -- ska_pst_smrb_info

To run the core:

make local-docker-run -- ska_pst_smrb_core [args]

To run the reader:

make local-docker-run -- ska_pst_smrb_reader [args]

To run the writer:

make local-docker-run -- ska_pst_smrb_writer [args]

Example to run the test mode of core with a given config file:

make local-docker-run -- ska_pst_smrb_core -t -v -f /mnt/ska-pst-smrb/src/apps/data/config.txt

Helm Charts

A Helm chart consists of template files that are rendered based on the contents of a payload, i.e. values.yaml

Helpers

  1. Render kubernetes manifest templates using default values from values.yaml

➜  ska-pst-smrb git: (at3-231-establish-smrb-helm-chart) ✗ make local_helm_render
---
# Source: ska-pst-smrb/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: release-name-ska-pst-smrb
...

  1. Display contents of values.yaml

➜  ska-pst-smrb git: (at3-231-establish-smrb-helm-chart) ✗ make local_helm_values 
---
# Default values for ska-pst-smrb.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

replicaCount: 1

smrb:
  enabled: true
  kind: Pod
  envFrom:
    - type: "configMapRef"
      name: smrb
  volumeMounts: []
...

Manual k8s deployment

  1. Set variables in local PrivateRules.mak

# i.e use specific values.yaml for helm chart as configuration payload
K8S_CHART_PARAMS="--values=tests/integration/k8s-test/k8srunner-psi-low.yaml"
KUBE_NAMESPACE=demo

  1. Generate, review and deploy rendered manifest.yaml

#!/bin/bash
# Generate manifest.yaml
make k8s-template-chart > manifest.yaml

# Review manifest.yaml
cat manifest.yaml

# Deploy manifest.yaml
kubectl apply -f manifest.yaml

# Check deployment state
kubectl get -f manifest.yaml

# Grab ska-pst-smrb-reader container logs
kubectl logs pod/test-ska-pst-smrb-test-performance -c ska-pst-smrb-reader

Simulating component tests

Python driven kubernetes deployment through the python packages pytest, pytest_bdd and kubernetes.

  1. bdd_pytest - The make target bdd_k8s_pytest ingests the feature file smrb.feature.SMRB binary test which includes the following test scenarios.

    • Invalid ska_pst_smrb_core configuration

    • SIGINT received by ska_pst_smrb_core

    • SIGTERM received by ska_pst_smrb_core

    make bdd_pytest

  1. bdd_k8s_pytest - The make target bdd_k8s_pytest ingests the feature file k8s.feature. It consists of container behavior tests using raw kubernetes manifest file. The scenarios are:

    • Deploy pod with a valid configuration

    • reserve SMRB resources inside a container

    • delete SMRB resources inside a container

    make bdd_k8s_pytest

  1. bdd_redeploy - The make target bdd_redeploy testing of helm generated manifest file. It uses the payload bdd-component.yaml. The current bdd test involves the act of redeploying the ska_pst_smrb_core pod which proves that reserved resources are deleted upon the pods deletion and can be used by another pod for redeployment.

    make bdd_k8s_pytest

Known bugs

None

Troubleshooting

None

License

See the LICENSE file for details.