ska_sdp_batchlet.utils.dask_cluster.factory module

class ska_sdp_batchlet.utils.dask_cluster.factory.DaskClusterFactory

Bases: object

A factory class which returns a new instance of a SpecCluster based dask cluster instance.

static get_cluster(dask_params)

Create and return a dask cluster based on the execution environment.

The selection logic is as follows:

1. If inside a slurm job allocation, return DaskSlurmCluster

2. Else, return a DaskLocalCluster.

Parameters:

dask_params (dict) -- User provided dask parameters. Given to batchlet as json input, in the key "dask_params".

Return type:

SpecCluster

Returns:

Cluster instance which inherits SpecCluster.