BPP

The Batch Preprocessing Pipeline (BPP) applies a configurable chain of transformations to visibility data read from storage, in order to prepare:

  1. Calibrator observations for the instrumental calibration (INST) pipeline

  2. Target observations for the self-calibration (ICAL) pipeline

Case 1: Preparing Calibrator observations

        flowchart LR
    storedvis[("Stored Visibilities<br>(Calibrator)")] --> sf
    subgraph bpp["Batch Pre-Processing (BPP)"]
    sf["Static<br>flagging"] --> df["Dynamic<br>flagging"] --> bss["Bright source<br>subtraction"]
    end
    bss --> outvis[("Pre-Processed<br>Visibilities<br>(Calibrator)")]
    outvis --> inst(("Instrumental<br>Calibration<br>(INST)"))
    inst --> caltable[("Gain Table<br>(Calibrator)")]

BPP prepares calibrator observations for INST, which includes flagging bad visibilities and optionally subtracting bright, out-of-field sources which would degrade the effectiveness of other batch pipelines.

Case 2: Preparing Target observations

        flowchart LR
    storedvis[("Stored Visibilities<br>(Target)")] --> sf
    subgraph bpp["Batch Pre-Processing (BPP)"]
    sf["Static<br>flagging"] --> df["Dynamic<br>flagging"] --> bss["Bright source<br>subtraction"] --> ac["Applycal"] --> avg["Averaging"]
    end
    avg --> outvis[("Pre-Processed<br>Visibilities<br>(Target)")]
    outvis --> ical(("Self-Calibration<br>(ICAL)"))
    caltable[("Gain Table<br>(Calibrator)")] --> ac

BPP also prepares target field observations for ICAL to process. This additionally involves:

  1. Applying to the target field visibilities the gains derived by INST on the most recent calibrator observation.

  2. Averaging the visibilities in time and frequency to reduce the compute cost of ICAL.