SKA-PST
This project provides the helm chart that includes the RECV.CORE, SMRB.CORE, and DSP.DISK software components of the Square Kilometre Array (SKA)’s Pulsar Timing (PST) instrument.
Architecture
Overview
The purpose of the PST (Pulsar Timing) is to process up to 16, dual-polarisation, oversampled, tied-array beams (from here on simply referred to as “beams”) formed in the SKA1 beamformers (CSP_Mid.CBF and CSP_Low.CBF), producing refined data products to be used in the SKA1 Science Data Processors (SDP_Mid and SDP_Low). The type of data product generated by the PST will depend on its configuration, with three specific observation modes supported: Pulsar Timing, Dynamic Spectrum and Flow Through. The observation mode is set on a per-beam basis and as such the PST is capable of producing multiple different types of data product simultaneously.
Artefacts
All of PST software components are built and packaged into artefacts as advised by the SKAO Developer Portal. The software binaries are packaged into OCI images while their deployment configurations are packaged into Helm artefacts.
OCI IMAGES
OCI Image Name |
OCI Image Registry |
Source Code Repository |
---|---|---|
ska-pst-dsp |
||
ska-pst-lmc |
||
ska-pst-recv |
||
ska-pst-smrb |
Helm Charts
The ska-pst
is an umbrella chart consisting of ska-pst-core
, and ska-pst-lmc
sub-charts. See each of the components documentation pages for further information.
Software Components
PST Component Name |
Short Description |
Documentation link |
---|---|---|
DSP |
Digital signal processing pipeline |
|
LMC |
Local Monitoring and Control |
|
RECV |
Data acquisition system |
|
SMRB |
Shared memory ring buffer manager |
Requirements
Kubernetes and Helm must be installed in order to run SKA PST.
Kubernetes
There are a number of ways to install and run Kubernetes as a local development environment. The one supported by the SKA is Minikube.
Helm
Helm is a kubernetes package manager. For its installation, see Installing Helm.
K9s
The default kubernetes client interface is kubectl. To simplify the interaction with kubernetes, K9s can be used. For more information, see k9scli.
Helm
This is a summary of the Helm chart parameters that can be used to customise the PST helm charts. The defaults values can be located in the chart’s values file.
SKA PST
Global values
Global values are used to propagate configuration to ska-pst-core and ska-pst-lmc.
Parameter |
Description |
Default |
---|---|---|
|
PST.CORE components |
|
|
RECV.CORE components. |
|
|
Port configurations used for communication with PST.LMC |
|
|
Port exposed to outside of the k8s pod |
18080 |
|
Protocol |
TCP |
|
Port exposed by a container in a k8s Pod |
18080 |
|
Port configurations used to recieve UDP data stream. |
|
|
Port exposed to outside of the k8s pod |
32080 |
|
Protocol |
UDP |
|
Port exposed by a container in a k8s Pod |
32080 |
Override example:
# Override ports used for communication between ska-pst-core and ska-pst-lmc
global:
tango_host: databaseds-tango-base-test:10000
sub-system:
ska-tango-base:
enabled: false
ports:
recv:
recv-mgmt:
port: 28080
smrb:
smrb-mgmt:
port: 28081
dsp:
dsp-mgmt:
port: 28082
SKA PST CORE Configurations
Parameter |
Description |
Default |
---|---|---|
|
Command executed in the dsp container |
bash |
|
Arguments to the command |
["-c","ska_pst_dsp_disk -v -c ${DSP_MGMT_PORT}"] |
|
Command executed in the recv container |
bash |
|
Arguments to the command |
["-c","ska_pst_smrb_core -v -c ${SMRB_MGMT_PORT}"] |
|
Command executed in the smrb container |
bash |
|
Arguments to the command |
["-c","ska_pst_recv_udpdb ${POD_IP} -v -p ${RECV_DATASTREAM_PORT} -c ${RECV_MGMT_PORT}"] |
|
List of storage configurations mounted to the dsp container of ska-pst-core |
[{ containerPath: "/tmp/", name: "default", size: "500Mi", type: "emptyDir" }] |
|
Used for reserving k8s nodes resources. |
|
Override example:
ska-pst-core:
enabled: true
# Override commands + args to bash to all containers in ska-pst-core
recv:
command: ["bash"]
args: []
smrb:
command: ["bash"]
args: []
dsp:
command: ["bash"]
args: []
# Use all available storage types in SKAO infrastructure
# https://developer.skao.int/en/latest/tools/containers/orchestration-guidelines.html#storage-in-kubernetes-clusters-managed-by-the-systems-team
volumeMounts:
- containerPath: /tmp/
name: default
size: 50Gi
type: emptyDir
- containerPath: /mnt/hostPath
hostPath: /tmp/
name: mnt-hostPath
type: hostPath
- containerPath: /mnt/block
name: mnt-block
type: block # renders to k8s templates required for utilising storage class bds1
size: 50Gi
- containerPath: /mnt/nfs
name: mnt-nfs
type: nfs # renders to k8s templates required for utilising storage class nfss1
size: 50Gi
# Deploy to specific host. i.e. psi-pst1 in PSI LOW
nodeSelector:
kubernetes.io/hostname: psi-pst1
# Request for specific k8s node resources
limits:
cpu: 1000m
memory: 16Gi
requests:
cpu: 500m
memory: 8Gi
SKA PST LMC Configurations
Detailed PST.LMC documentation is located at SKA PST LMC. Default ska-pst-lmc chart values are located at GitLab
Override example:
# Disable simulation mode
# Override PST.CORE component api endpoints to ska-pst-core
ska-pst-lmc:
enabled: true
beam:
simulationMode: 0
dsp:
simulationMode: 0
dsp_process_api_endpoint: ska-pst-core
recv:
simulationMode: 0
recv_process_api_endpoint: ska-pst-core
smrb:
simulationMode: 0
smrb_process_api_endpoint: ska-pst-core
SKA PST
Components
The ska-pst is an umbrella chart consisting of ska-pst-core, and ska-pst-lmc sub-charts.
Installation
Prerequisites
# Add SKA's helm repository
helm repo add skao https://artefact.skao.int/repository/helm-internal
# Search repository for ska-pst chart
helm search repo skao/ska-pst
# Store defaults to file
helm show values skao/ska-pst > values.yaml
# modify values (optional)
vi values.yaml
# Render template to provided values (optional)
## Used to validate if values are rendered to emplates as expected
make k8s-template-chart
## On the scenario that the SKA cicd makefile pipeline machinery isnt used.
helm template skao/ska-pst
Deployment
# Prepare environment variables (optional)
## Declare k8s namespace
export KUBE_NAMESPACE=pst
## Declare target application for wait
export KUBE_APP=ska-pst-lmc
# Install chart
## It can be installed using the SKA cicd makefile pipeline machinery.
make k8s-install-chart
## Environment variables can be used if overrides are required.
make K8S_CHART_PARAMS="--values=value.yaml" k8s-install-chart
## On the scenario that the SKA cicd makefile pipeline machinery isnt used.
helm install --generate-name skao/ska-pst
## And if overrides are desired
helm install --values=value.yaml --generate-name skao/ska-pst
# Wait for pods to be in a ready or completed state (optional)
## Use SKA cicd makefile pipeline machinery
make k8s-wait
## On the scenario that the SKA cicd makefile pipeline machinery isnt used.
kubectl -n ${KUBE_NAMESPACE} wait pod --for=condition=ready -l app=${KUBE_APP} --timeout=360s
SKA PST CORE STANDALONE
Overview
The ska-pst-core
chart is used for deploying PST.CORE. It is used as a dependency chart by the ska-pst
chart.
Components
PST.CORE consists of RECV.CORE, SMRB.CORE, and DSP.DISK. See each of the components documentation pages for further information.
Installation
Prerequisites
# Add SKA's helm repository
helm repo add skao https://artefact.skao.int/repository/helm-internal
# Search repository for ska-pst-core chart
helm search repo skao/ska-pst-core
# Store defaults to file
helm show values skao/ska-pst-core > values.yaml
# modify values (optional)
vi values.yaml
# Render template to provided values (optional)
## Used to validate if values are rendered to emplates as expected
make k8s-template-chart
## On the scenario that the SKA cicd makefile pipeline machinery isnt used.
helm template skao/ska-pst-core
Deployment
# Prepare environment variables (optional)
## Declare k8s namespace
export KUBE_NAMESPACE=pst
## Declare helm chart to be installed
export K8S_CHART=ska-pst-core
## Declare target application for wait
export KUBE_APP=ska-pst-core
# Install chart
## It can be installed using the SKA cicd makefile pipeline machinery.
make k8s-install-chart
## Environment variables can be used if overrides are required.
make K8S_CHART_PARAMS="--values=value.yaml" k8s-install-chart
## On the scenario that the SKA cicd makefile pipeline machinery isnt used.
helm install --generate-name skao/ska-pst-core
## And if overrides are desired
helm install --values=value.yaml --generate-name skao/ska-pst-core
# Wait for pods to be in a ready or completed state (optional)
## Use SKA cicd makefile pipeline machinery
make k8s-wait
## On the scenario that the SKA cicd makefile pipeline machinery isnt used.
kubectl -n ${KUBE_NAMESPACE} wait pod --for=condition=ready -l app=${KUBE_APP} --timeout=360s
Troubleshooting
Override files
To confirm compatibility of the values.yaml
used for overriding the defaults, the following command can be used to confirm if the templates are rendered as expected.
helm template skao/ska-pst --values=values.yaml
Container logs
To specifically display the logs of all containers, application labels can be used to simplify the k8s command.
# Display logs of all containers in all pods that has the label ska-pst-core
kubectl -n pst logs -l app=ska-pst-core
# Display logs of all containers in all pods that has the label ska-pst-lmc
kubectl -n pst logs -l app=ska-pst-lmc
Automated Tests
Deployment
There are 3 charts present in the ska-pst
repository. The test-parent
, ska-pst
and ska-pst-core
.
Each of the chart undergoes a series of deployment tests using GitLab’s CICD pipelines
Scan
Data Generator
The main tool for generating data is ska_pst_recv_udpgen
which is an output artefact of the ska-pst-recv repository
It is imported in the ska-pst-testutils
oci image which resides in the ska-pst
repository.
Simulating a Scan
The scan simulation is executed through the SKA cicd makefile pipeline machinery target k8s-test
.
The underlying code involves the usage of pytest and tango to interact with ska-pst-lmc
and ska-pst-core
.
The simulation can be executed through the command make k8s-test PYTHON_VARS_TEST_SUITE=$TESTSUITE
.
The default test suite is integration
.
Test Suite(s)
At this time of writing, there are currently two test suites. integration
and invalidpackets
tests.
The tests are yet to be integrated with Jira and XRay where the test payload are in the form of Gherkin feature files to be ingested by pytest.
Sample Scan configuration JSON
Detailed description of the JSON schema has yet to be documented. The base configuration as well as overrides used for the scan simulation test can be found here.
Integration Tests
Single Scan
Multiple Scans
Abort
Abort before receiving packets
Negative Tests
Invalid Packets
Performance Tests
Performance related tests involving data transmission relies on the options available to ska_pst_recv_udpgen
. The available options include the transmission rate (in Gb/s) and data transmission duration (in seconds).