Benchmarking Pipeline

We have integrated various ways into this pipeline which can be used to benchmark it.

With dask performance report

You can generate detailed reports of dask's run using dask diagnostics.

When running the pipeline in distributed mode, you can pass --with-report option which store the dask report in dask_report.html file inside the specified output directory.

ska-sdp-spectral-line-imaging run --input input.ps --config config.yml --output-path output/benchmark --dask-scheduler localhost:8786 --with-report

Note that pipeline must be run with a dask scheduler for reports to be generated.

Also, since dask performance report captures detailed metrics per task (so per process), this might affect the computation times.