Dask

Dask allows distributed computation in Python.

Chart Details

This chart will deploy the following:

1 x Dask scheduler with port 8786 (scheduler) and 8787 (Web UI) exposed on a ClusterIP (default)
2 x Dask workers that connect to the scheduler
1 x Jupyter lab notebook (optional, false by default) with port 8888 exposed on a ClusterIP (default)
All using Kubernetes Deployments

Note: only version 0.2.0 of the chart contains the Jupyter lab notebook pod.

Tip: See the Kubernetes Service Type Docs for the differences between ClusterIP, NodePort, and LoadBalancer.

Installing the Chart

First we need to add the helmdeployer-chars repo to our local helm config.

helm repo add ska-sdp-helm https://gitlab.com/ska-telescope/sdp/ska-sdp-helmdeploy-charts/-/raw/master/chart-repo
helm repo update

To install the dask chart with the release name test:

helm install test ska-sdp-helm/dask

Depending on how your cluster was set up, you may also need to specify a namespace with the following flag: --namespace my-namespace.

Default Configuration

The following tables list the configurable parameters of the Dask chart and their default values. Note: the container images are not provided by default, you have to specify them in a custom values.yaml file or as a command line argument.

Dask scheduler

Parameter	Description	Default
`scheduler.name`	Dask scheduler name	`scheduler`
`scheduler.image`	Container image name	`""`
`scheduler.imageTag`	Container image tag	`""`
`scheduler.replicas`	k8s deployment replicas	`1`
`scheduler.tolerations`	Tolerations	`[]`
`scheduler.nodeSelector`	nodeSelector	`{}`
`scheduler.affinity`	Container affinity	`{}`

Dask webUI

Parameter	Description	Default
`webUI.name`	Dask webui name	`webui`
`webUI.servicePort`	k8s service port	`80`
`webUI.ingress.enabled`	Enable ingress controller resource	false
`webUI.ingress.hostname`	Ingress resource hostnames	dask-ui.example.com
`webUI.ingress.tls`	Ingress TLS configuration	false
`webUI.ingress.secretName`	Ingress TLS secret name	`dask-scheduler-tls`
`webUI.ingress.annotations`	Ingress annotations configuration	null

Dask worker

Parameter	Description	Default
`worker.name`	Dask worker name	`worker`
`worker.image`	Container image name	`""`
`worker.imageTag`	Container image tag	`""`
`worker.replicas`	k8s hpa and deployment replicas	`2`
`worker.resources`	Container resources	`{}`
`worker.tolerations`	Tolerations	`[]`
`worker.nodeSelector`	nodeSelector	`{}`
`worker.affinity`	Container affinity	`{}`
`worker.port`	Worker port (defaults to random)	`""`

Jupyter

Parameter	Description	Default
`jupyter.name`	Jupyter name	`jupyter`
`jupyter.enabled`	Include optional Jupyter server	`false`
`jupyter.image`	Container image name	`""`
`jupyter.imageTag`	Container image tag	`""`
`jupyter.replicas`	k8s deployment replicas	`1`
`jupyter.servicePort`	k8s service port	`80`
`jupyter.resources`	Container resources	`{}`

Custom Configuration

If you want to change the default parameters, you can do this in two ways.

YAML Config Files

You can update the default parameters in values.yaml by creating your own custom YAML config file with the updated parameters, and specify this file when installing your chart with the -f flag. Example:

helm install test ska-helm/dask -f values.yaml

Command-Line Arguments

If you want to change parameters for a specific install without changing values.yaml, you can use the --set key=value[,key=value] flag when running helm install, and it will override any default values. Example:

helm install test ska-helm/dask --set jupyter.enabled=false

Changelog

0.2.1

updated the dask worker-deployment template to allow for custom volumes and volume claims

0.2.0

Jupyter lab notebook deployment added to chart