Troubleshooting
When things go wrong, its best to start by checking on the status of the pods and
investigating the logs of the various processes that operate within PST. The following
kubectl
commands assume that either PST is running in the default namespace or
the namespace has been configured with
kubectl config set-context --current --namespace=$KUBE_NAMESPACE
Pod status
To display the status of the pods
# Display a quick summary of all pods
kubectl get pods
# Display detail of the pod running the PST core applications
kubectl describe pods test-ska-pst-core
# See if any applications have exited unexpectedly
kubectl describe pods test-ska-pst-core | grep Exit -A 5 -B 5
To run a bash
shell on a specific container
# Run bash on the PST DSP container
kubectl exec -ti test-ska-pst-core -c dsp -- bash
# Run bash on the PST JupyterLab container
kubectl exec -ti test-ska-pst-jupyterlab -- bash
Container logs
To display the logs of all containers, application labels can be used to simplify the k8s command.
# Display logs of all containers in all pods that has the label ska-pst-core
kubectl logs -l app=ska-pst-core
# Display logs of all containers in all pods that has the label ska-pst-lmc
kubectl logs -l app=ska-pst-lmc
To display the logs of specific containers within a pod:
# Display logs of PST RECV
kubectl logs test-ska-pst-core -c recv
# Display logs of PST SMRB
kubectl logs test-ska-pst-core -c smrb
# Display logs of PST DSP
kubectl logs test-ska-pst-core -c dsp
# Display logs of PST SEND
kubectl logs test-ska-pst-core -c send
# Display logs of PST STAT
kubectl logs test-ska-pst-core -c stat
Override files
To confirm compatibility of the values.yaml
used for overriding the defaults, the following command can be used to confirm if the templates are rendered as expected.
helm template skao/ska-pst --values=values.yaml