.. _diagnostic_products: ************************ Diagnostic Data Products ************************ This page describes diagnostic data products generated by the pipeline. RFI Flagging Report =================== The RFI Flagging Report summarises the fraction of visibility data flagged as radio-frequency interference in each output dataset, across time, baseline, and frequency. Reports are saved as `xarray datasets `_, which are self-descriptive collections of labeled multi-dimensional arrays that share dimensions and coordinate axes. For each input visibility dataset, a corresponding report is saved in the main pipeline output directory as ``_flagging_report.zarr``. A corresponding plot of the report is saved as a PNG file as ``_flagging_report.png``. Standalone CLI app ------------------ It is also possible to generate a flagging report for any Measurement Set independently of the main pipeline, using the ``ska-sdp-flagging-report`` command: .. code-block:: bash ska-sdp-flagging-report path/to/dataset.ms This saves ``dataset_flagging_report.zarr`` and ``dataset_flagging_report.png`` in the current working directory. The app uses a local Dask cluster sized to the number of available CPU cores. Example Plot ------------ Here is an example plot of a flagging report as saved by the pipeline, after a run on a small MeerKAT dataset. Note that you can generate your own plots from the xarray dataset, see below. .. Note: in the target path below, the leading underscore in "_images" is required .. to refer to the correct build output directory. .. image:: ./images/flagging_report_example.png :alt: Example flagging report plot :align: center :target: ./_images/flagging_report_example.png Loading and working with reports -------------------------------- All you need is the ``xarray`` library to load and inspect flagging reports. For example: .. code-block:: python import xarray as xr report = xr.open_zarr("mydataset_flagging_report.zarr", chunks=None) print(report) This should print something similar to: .. code-block:: none Size: 3MB Dimensions: (baseline_id: 1953, frequency: 64, time: 224) Coordinates: baseline_antenna1_name (baseline_id)