PSS Receive

PSS receive is an SDP component whose purpose is to receive data from the PSS pipeline. PSS searches beamformer data for compelling pulsar and single pulse candidates. It exports those candidates over a spead2 stream to pss-receive. Currently pss-receive stores single pulse candidate metadata. In the future, it will be updated to receive pulsar candidate data. These instructions describe the pss-receive helm chart and its deployment.

Setting up minikube

The following has been tested on Scientific Linux 7.9 using

  • minikube v1.21.0

  • Kubernetes v1.19.0

  • Docker v20.10.6

Start minikube, using a virtualbox driver and the Calico container network interface:

minikube start --cpus=16 --memory=32g --driver virtualbox --disk-size=40g --container-runtime=docker --wait=apiserver --kubernetes-version=1.19.0 --cni=calico

Configure minikube host kernel parameters:

minikube ssh <<EOT
sudo sysctl net.core.rmem_max=268435456
sudo sysctl net.core.wmem_max=268435456
sudo sysctl net.core.netdev_max_backlog=65536
sudo sysctl net.core.wmem_default=268435456
sudo sysctl net.core.rmem_default=268435456

Update calico to use larger MTU (maximum transmission unit):

kubectl patch configmap/calico-config -n kube-system --type merge -p '{"data":{"veth_mtu": "9000"}}'
kubectl rollout restart daemonset calico-node -n kube-system

Create pss namespace:

kubectl create namespace pss

Deploying pss-receive

The pss-receive kubernetes manifest consists of the pss-receive job (which deploys the correct container image, and opens up the ports (defined in the helm chart), a network service which routes incoming data to the pss-receive pod, and a persistent volume (and persistent volume claim) for storage of the received candidate data.

After cloning the SDP Helm Deployer Charts repository,

cd ska-sdp-helmdeploy-charts/charts

Install the pss-receive helm chart (this will take ~40 minutes):

helm install test pss-receive -n pss --set helmdeploy.createClusterRole=true

The deployment can be monitored by running:

kubectl get all -n pss

NAME                    READY   STATUS    RESTARTS   AGE
pod/pss-receive-v29tx   1/1     Running   0          80m

NAME                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/pss-receive   ClusterIP   <none>        9021/UDP   80m

NAME                    COMPLETIONS   DURATION   AGE
job.batch/pss-receive   0/1           80m        80m

which shows the pss-receive pod and the network service. Once the pss-receive pod enters the running state, we can check if the container has been deployed correctly by running:

kubectl logs pss-receive-v29tx -n pss

which, if the pss-receive application has been successfully launched should show:

[debug][tid=140659546208128][/opt/pss-pipeline/thirdparty/cheetah/cheetah/../cheetah/exporters/detail/DataExport.cpp:61][1629726512]Creating sink of type sp_candidate_data (id=candidate_files)