Data Product Definitions
========================

The Continuum Imaging Pipeline (CIMG) produces a number of data products as outputs depending on the
imaging mode requested in the configuration file. The intended Observatory Data Product (ODP)
definitions are captured in the following `Solution Intent page on Confluence
<https://confluence.skatelescope.org/pages/viewpage.action?pageId=353340247>`_, and the current
outputs in the sections below.

The `pipeline configuration file
<https://gitlab.com/ska-telescope/sdp/science-pipeline-workflows/ska-sdp-cimg/-/blob/main/config/cimg_config.yaml?ref_type=heads>`_
has a number of parameters that control the data products generated. For example, the target folder
for the data products is determined by the ``--output-dir`` CLI parameter. The
``image_settings.name`` parameter in the configuration file determines the prefix for the data (by
default ``"cimg_odp"``).

Image Data Products
-------------------

The main data products generated by the pipeline are summarised in the table below. For each
frequency channel imaged, as well as for the full observation combined across all channels (MFS),
the following FITS files are produced:

.. list-table::
    :header-rows: 1
    :widths: 35 35 30

    - - File Name (per channel)
      - File Name (combined channels)
      - Description
    - - ``cimg_odp-00xx-dirty.fits``
      - ``cimg_odp-MFS-dirty.fits``
      - Dirty image
    - - ``cimg_odp-00xx-image.fits``
      - ``cimg_odp-MFS-image.fits``
      - Clean image
    - - ``cimg_odp-00xx-image-pb.fits``
      - ``cimg_odp-MFS-image-pb.fits``
      - Clean image with primary beam correction applied
    - - ``cimg_odp-00xx-model.fits``
      - ``cimg_odp-MFS-model.fits``
      - Model image (flat noise)
    - - ``cimg_odp-00xx-model-pb.fits``
      - ``cimg_odp-MFS-model-pb.fits``
      - Model image with primary beam correction applied
    - - ``cimg_odp-00xx-model-fpb.fits``
      - NA
      - Model images for facet based prediction
    - - ``cimg_odp-00xx-residual-pb.fits``
      - ``cimg_odp-MFS-residual-pb.fits``
      - Residual of beam corrected clean image and model image
    - - ``cimg_odp-00xx-residual.fits``
      - ``cimg_odp-MFS-residual.fits``
      - Residual of flat noise image and model
    - - ``cimg_odp-00xx-beam.fits``
      - NA
      - Represents the Stokes I response: a combination of the average beam and average gain
        solution corrections.

In the above ``00xx`` represents the filename for each frequency channel. For example, for a run
requesting 12 frequency channels there will be 12 such files, numbered from ``0000`` through
``0011``.

Direction-dependent PSF Files
-----------------------------

In addition to the files above, for each frequency channel, wsclean calculates direction dependent
PSFs on an 8 by 8 grid across the image (the number of grid tiles may be made configurable in a
future release). More detail concerning this is available from the `wsclean documentation
<https://wsclean.readthedocs.io/en/latest/direction_dependent_psfs.html>`_. For a run using 12
frequency channels, for example, these PSF files are available as:

    - ``cimg_odp-d0000-0000-psf.fits``
    - ...
    - ``cimg_odp-d0000-0011-psf.fits``
    - ...
    - ``cimg_odp-d0063-0000-psf.fits``
    - ...
    - ``cimg_odp-d0063-0011-psf.fits``

where ``d0000`` to ``d0063`` represent the 64 tiles in the 8 by 8 grid, and ``0000`` to ``0012``
represent the frequency channels.

CLEAN component lists
---------------------

As part of the imaging process, wsclean produces a source catalog in `makesourcedb
<https://www.astron.nl/lofarwiki/doku.php?id=public:user_software:documentation:makesourcedb#makesourcedb>`_
format containing a detailed component list of the sources used in the model of the final image. The
catalog is available as: ``cimg_odp_sources.txt``. In addition, the ``cimg_odp_sources-pb.txt`` file
contains the primary beam corrected source components. Further details can be found in the `wsclean
component list documentation <https://wsclean.readthedocs.io/en/latest/component_list.html>`_.