Plasma Pipeline

The plasma-pipeline chart deploys its processes using separate containers in the same pod. That guarantees they will be on the same node and be able to share the memory volume. A StatefulSet with replicas > 1 would create multiple copies of the pod, so that would scale up to multiple nodes.

This chart:

  • Creates a Job

  • Pod contains four containers:

    • Plasma store process

    • sender

    • receiver (writes to plasma store)

    • MS writer (reads from plasma store)

  • Commands are hard-coded

  • Two volumes:

    • Memory emptyDir for the Plasma store. This is mounted by the Plasma store process, receiver and MS writer persistent volume claim (PVC) for data. This is not created by the template. This is mounted by the sender, receiver and MS writer

Deploying plasma pipeline

Start minikube

minikube start --vm-driver=virtualbox

Create namespace

kubectl create namespace plasma

After cloning the SDP Helm Deployer Charts repository,

cd ska-sdp-helmdeploy-charts/charts

Install the chart

helm install plasma-test plasma-pipeline -n plasma

The deployment can be monitored using k9s or by running

kubectl get all -n plasma

NAME                                           READY   STATUS    RESTARTS   AGE
pod/plasma-test-plasma-pipeline-plasma-thg85   0/4     Pending   0          92s

NAME                                           COMPLETIONS   DURATION   AGE
job.batch/plasma-test-plasma-pipeline-plasma   0/1           92s        92s