SKA SDP end-to-end batch continuum imaging pipeline

This repository contains tools and scripts needed to run SKA SDP End-to-End Batch Continuum Imaging Pipeline (more oftenly referred as the "batch e2e" pipeline) on any infrastructure. The "batch e2e" pipeline has multiple stages, each stage uses different pipelines developed by different teams at SKA SDP. The following image shows overview of the flow of the "batch e2e" pipeline:

_images/e2e_overview_redesigned.drawio.svg

The end-to-end pipeline can be run on slurm using either:

  1. CLI : Recommended way using a single CLI entrypoint and YAML configuration as input

  2. Batch Pipeline Scripts : Older way using pure bash-based slurm scripts

Readme

Releases

Contents

API