Stages and configurations

The descriptions of each stage are copied from the docstrings of stages. Each stage has parameters, which are defined in the YAML config file passed to the pipeline.

This configuration is used by the ska-sdp-batch-e2e-pipeline run command.

Stage configurations

initialise_sdm

Initialise an empty Science Data Model (SDM) and populate it with sky model files.

Name	Type	Default	Description	Nullable	Allowed Values
sky_model_paths	`list`[`SkyModelInputPath`]	<PIPER_UNDEFINED>	List of sky model paths, where each entry relates to a single field. Each entry must have: * path: list of file paths to sky model files for this field. * field_id: field id corresponding to these sky model files. * dest_file_name: file name for the input file path provided. e.g. sky_model_paths: - path: /path/to/sky_model.csv field_id: field_cal dest_file_name: sky_model.csv	False

instrumental_calibration

Performs instrumental calibration.

Name	Type	Default	Description	Nullable
calibrator_sources	`list`[`CalibratorSource`]	<PIPER_UNDEFINED>	List of calibrator source dictionaries, where each dictionary maps to a single calibrator source. Each dictionary should have keys: * measurement_sets: paths to the calibrator Measurement Sets (MS) * field_id: Field id corresponding to the source. This follows the pydantic schema defined by `CalibratorSource`.	False
averager	`AveragerStep`	AveragerStep(timestep=4, freqstep=4)	It is used to average the visibility data.	False
preflagger	`PreFlaggerStep`	PreFlaggerStep(freqrange='[154.250..159.810]')	It is used to pre-flag the visibility data.	False
bandpass_calibration	`BandpassCalibration`	BandpassCalibration(solver='jones_substitution', niter=50, tol=0.001)	Bandpass calibration configuration parameters	False

target_calibration

Perform the target calibration stage, including target BPP and self-calibration.

Name	Type	Default	Description	Nullable
target_sources	`list`[`TargetSource`]	<PIPER_UNDEFINED>	List of target source dictionaries, where each dictionary maps to a single target source. Each dictionary should have keys: * measurement_sets: paths to the target Measurement Sets (MS) * field_id: Field id corresponding to the source. This follows the pydantic schema defined by `TargetSource`.	False
averager	`AveragerStep`	AveragerStep(timestep=4, freqstep=4)	Averager parameters. It is used to average the visibility data.	False
preflagger	`PreFlaggerStep`	PreFlaggerStep(freqrange='[154.250..159.810]')	It is used to pre-flag the visibility data.	False
complex_gain_calibration	`ComplexGainCalibration`	ComplexGainCalibration(niter=500, tol=1e-06)	Configuration for the Instrumental Target calibration pipeline.	False
ionospheric_calibration	`IonosphericCalibration`	IonosphericCalibration(niter=500, tol=1e-06, zernike_limit=None)	Configuration for the Target Ionospheric calibration pipeline	False
self_calibration	`SelfCalibration`	SelfCalibration(min_selfcal_loops=4, max_selfcal_loops=8)	Configuration for the self-calibration sub-pipeline.	False
ical_di_calibration	`SelfCalibration` \| `NoneType`	SelfCalibration(min_selfcal_loops=4, max_selfcal_loops=8)	Configuration for DI calibration using ICAL	True
calibrator_field_id	`str` \| `NoneType`	None	Field id corresponding to the calibrator source. If Field id is Not passed then calibrator_field_id from upstream output will be used.	True
use_ical_di	`bool`	False	Set as true to use ICAL DI calibration If set to false, INST target calibration will be used instead.	False

continuum_imaging

Perform the continuum imaging stage.

Name	Type	Default	Description	Nullable
imaging_params	`ContinuumImaging`	ContinuumImaging(polarizations='stokes_i', image_size=(4000, 4000), pixel_scale=0.0005555555555555556, briggs_robust_weighting=-0.5, output_channels=12, fit_spectral_pol=3, gaussian_taper=0.0)	Continuum imaging parameters.	False
target_sources	`list`[`TargetSource`] \| `NoneType`	None	Target sources. Its a list of dictionary, where each dictionary relates to a single target. Each dictionary must have a key, "measurement_sets" (list of path to each scan of a target). If not provided, pre-processed target sources from upstream target_calibration stage will be used. e.g. target_sources: - measurement_sets: - target_source1.ms - target_source2.ms	True
fits_mask	`str` \| `NoneType`	None	Path to the fits mask provided by the user	True
facet_region	`str` \| `NoneType`	None	Path to the facet region file either produced from previous stage target calibration's ICAL or provided by the user.	True
solutions	`str` \| `NoneType`	None	Path to the solutions h5 either produced from previous stage target calibration's ICAL or provided by the user.	True