.. _qa_plots:

MCCS Calibration Quality Assurance Plots
========================================

This package is used to generate calibration solutions for the calibration
database. Metrics and plots that probe data quality and observing conditions
can help to indicate whether or not a given set of solutions is suitable.
Functions to generate summary plots are discussed below. An overview of simple
summary metrics is given on the :ref:`QA metrics <qa>` page.

The functions discussed on this page are intended to be called alongside
:py:func:`~ska_low_mccs_calibration.calibration.calibrate_mccs_visibility`,
using many of the same input or output data products. These functions return
matplotlib `figure`_ identifiers that can be displayed from the script or
notebook, or output as an image file in a standard format.

For example

.. code-block:: python

  import matplotlib.pyplot as plt
  from ska_low_mccs_calibration.qa import generate_closure_comparison_plots
  fig = generate_closure_comparison_plots(
      vis, modelvis, masked_antennas, min_uv=10, ctype="uvmax"
  )
  fig.savefig("closure_comparison.png")
  plt.show

One or more of these functions can be called automatically in
:py:func:`~ska_low_mccs_calibration.calibration.calibrate_mccs_visibility`
using optional dictionary :code:`qa_plots`. Any plotting functions from
:py:func:`~ska_low_mccs_calibration.qa` can be listed with a True or False
value, and any set to true will be called with PNG output (named “function.png”
without the `generate_` prefix). Use
:code:`print(ska_low_mccs_calibration.qa.__all__)` for a list. For example:

.. code-block:: python

  qa_plots = {
      "generate_gaintable_plots": True,
      "generate_visibility_comparison_plots": False,
      "generate_closure_comparison_plots": True,
  }

Gaintable plots
---------------

Function :py:func:`~ska_low_mccs_calibration.qa.generate_gaintable_plots`
displays the amplitude and phase of each antenna calibration solution, with 
separate panels for diagonal and off-diagonal Jones matrix elements. In the
example below,
:py:func:`~ska_low_mccs_calibration.calibration.calibrate_mccs_visibility` was
called with :code:`jones_solve=False`, so the off-diagonal terms are all zero.

.. image:: img/gaintable_plots.png

Gaintable scatter plots
-----------------------

Function
:py:func:`~ska_low_mccs_calibration.qa.generate_gaintable_scatter_plots`
displays the same amplitude and phase data as
:py:func:`~ska_low_mccs_calibration.qa.generate_gaintable_plots`, as a scatter
plot at the antenna locations.

.. image:: img/gaintable_scatter_plots.png

Visibility plots
----------------
.. _vis_plots:

Function :py:func:`~ska_low_mccs_calibration.qa.generate_visibility_plots`
displays the amplitude and phase of each visibility sample as a function of
baseline length -- in general these will be the calibrated visibilities
returned by
:py:func:`~ska_low_mccs_calibration.calibration.calibrate_mccs_visibility`
-- along with model and residual visibilities. A separate set of panels is
shown for each polarisation. In the example below,
:py:func:`~ska_low_mccs_calibration.calibration.calibrate_mccs_visibility` was
called with the GSM and :code:`nside=16`. This is a low resolution model that
allows the solver to run faster, however model errors can be seen in the plots
for the longer baselines.

.. image:: img/visibility_plots.png

Visibility comparison plots
---------------------------

Function :py:func:`~ska_low_mccs_calibration.qa.generate_visibility_comparison_plots`
displays the same amplitude and phase data as
:py:func:`~ska_low_mccs_calibration.qa.generate_visibility_plots`, as a scatter
plot of visibility values versus model visibility values. Also shown are
histograms of the residual phase values.

.. image:: img/visibility_comparison_plots.png

Closure comparison plots
------------------------

Function :py:func:`~ska_low_mccs_calibration.qa.generate_closure_comparison_plots`
also displays scatter plots of visibility values versus model values, however
the data plotted are the closure triple products. As in
:py:func:`~ska_low_mccs_calibration.qa.generate_visibility_comparison_plots`,
histograms of the residual phase values are also shown.

In the example below, the scatter plots have been coloured by the maximum
baseline length in each triplet (running from blue to red), and a short baseline
cutoff of 10 metres has been used. The long-baseline errors that come from
using :code:`nside=16` can be seen as a wider model phase spread in red.

.. image:: img/closure_comparison_plots.png

If :py:func:`~ska_low_mccs_calibration.calibration.calibrate_mccs_visibility`
is rerun with :code:`nside=32`, the phases show better alignment. There is
extra structure in the YY products, which may be due to the higher power
levels seen in the :ref:`visibility plots <vis_plots>` above.

.. image:: img/closure_comparison_plots_gsm32.png

On the other hand, if
:py:func:`~ska_low_mccs_calibration.calibration.calibrate_mccs_visibility`
is rerun with :code:`skymodel="Sun"`, the sky is represented by a point source
and the closure phases for the model visibilties are close to zero. The small
offsets from zero are due to the Embedded Element Patterns in the modelled
visibilities.

.. image:: img/closure_comparison_plots_solar.png