Plasma Pipeline
The plasma-pipeline chart deploys its processes using separate containers in the same pod. That guarantees they will be on the same node and be able to share the memory volume. A StatefulSet with replicas > 1 would create multiple copies of the pod, so that would scale up to multiple nodes.
This chart:
Creates a Job
Pod contains four containers:
Plasma store process
sender
receiver (writes to plasma store)
MS writer (reads from plasma store)
Commands are hard-coded
Two volumes:
Memory emptyDir for the Plasma store. This is mounted by the Plasma store process, receiver and MS writer persistent volume claim (PVC) for data. This is not created by the template. This is mounted by the sender, receiver and MS writer
Deploying plasma pipeline
Start minikube:
minikube start --vm-driver=virtualbox
Create namespace:
kubectl create namespace plasma
After cloning the SDP Helm Deployer Charts repository,
cd ska-sdp-helmdeploy-charts/charts
Install the chart:
helm install plasma-test plasma-pipeline -n plasma
The deployment can be monitored using k9s or by running:
kubectl get all -n plasma
NAME READY STATUS RESTARTS AGE
pod/plasma-test-plasma-pipeline-plasma-thg85 0/4 Pending 0 92s
NAME COMPLETIONS DURATION AGE
job.batch/plasma-test-plasma-pipeline-plasma 0/1 92s 92s