Signal Display Metrics Generator
================================

This processor takes in the Full Visibilities, and outputs them based on
which metrics this processor is looking at, and the associated Flow entries.


Configuration
-------------

Configuring this processor is in 2 parts, first there are CLI parameters that
are given, and then there are also ConfigDB Flow entries that are also used.

ENV Variables
^^^^^^^^^^^^^

Some environment variables can be used to configure the processor.

.. list-table::
    :widths: auto
    :header-rows: 1

    * - Value
      - Default
      - Required
      - Comment

    * - ``SDP_KAFKA_HOST``
      - ``None``
      - No
      - This variable will only get fetched if the Config DB has been disabled.

    * - ``SDP_PB_ID``
      - ``pb-unknown``
      - Yes
      - This specifies which processing block to look at, without this variable
        other items will not be configured, for example flow entries. It isn't
        needed when using ``--use-random-ids``.

    * - ``SDP_EB_ID``
      - ``eb-unknown``
      - No
      - This will get overridden with the Config DB entry (if found)

    * - ``SDP_SUBARRAY_ID``
      - ``01``
      - No
      - This will get overridden with the Config DB entry (if found)


Command Line Options
^^^^^^^^^^^^^^^^^^^^

.. list-table::
    :widths: auto
    :header-rows: 1

    * - Value
      - Default
      - Required
      - Comment

    * - ``--use-random-ids``
      - ``False``
      - No
      - If set the IDs will be randomized, and no Config DB data will be used.

    * - ``--ignore-config-db``
      - ``False``
      - No
      - Never try connect to the Config DB (use defaults or ENV values for everything.)

    * - ``--disable-kafka``
      - ``False``
      - No
      - Send no data to Kafka if set.

    * - ``--metrics``
      - ``stats``
      - Yes
      - Set the comma-separated list of metrics this processor should care about.
        Besides metric names, the other options are ``stats``, and ``all``.


Flow Entries
^^^^^^^^^^^^

The expected flow entries should look like the following:

.. code-block:: json

    {
        "data_model": "MetricPayload",
        "sink": {
            "format": "msgpack_numpy",
            "host": "ska-sdp-kafka:9092",
            "kind": "data-queue",
            "topics": "metrics-amplitude-01"
        },
        "sources": [
            {
                "function": "SignalDisplayMetrics",
                "parameters": {
                    "metric_type": "amplitude",
                    "nchan_avg": 50,
                    "additional_windows": 5,
                    "rounding_sensitivity": 5
                },
                "uri": "metrics://signal-display/01/amplitude/low_res"
            }
        ]
    }

These flow entries should only be configured from ``ska-sdp-scripts``/``vis-receive``.

Flow Entry State
****************

The state of a flow entry is used to configure high-resolution sections of the
metric data.

A state entry should resemble the following:

.. code-block:: json

    {
        "windows": [
            {
                "start": 1000000,
                "end": 2000000,
                "channels_averaged": 1
            }
        ]
    }

Note that ``len(state["windows"])`` should be less than or
equal to ``flow["sources"][0]["parameters"]["additional_windows"]``.

Note that the state can also be ``None``, and the ``windows`` key can be missing.


Metrics Available
-----------------

The metrics that the processor can currently generate are listed below, with a
short description. For a better description refer to the full documentation on
the :external+ska-sdp-qa-display:doc:`Signal Display Documentation <userguide/graphs>`:

Stats
^^^^^

These are general stats regarding the current receive process. Currently you can get:

* The last known state of this receiver (most of the time this will be ``receiving``).
* The amount of payloads received (these can be aggregated).
* The amount of time slices received (multiple of these can be received per payload).
* The amount of seconds between this payload, and the previous payload (includes processing time).


Graphable Metrics
^^^^^^^^^^^^^^^^^

The following metrics are available for graphing and the documentation is
available in the :external+ska-sdp-qa-display:doc:`Signal Display Documentation <userguide/graphs>`:

* Spectrum
* Polarization vs Phase
* Polarization vs Amplitude
* Lag Plot
* Band Averaged Cross Correlation Power
* UV Coverage
* Phase variance as a function of uv/baseline distance
* Amplitude variance as a function of uv/baseline distance

The structure of these metrics are available in :external+ska-sdp-dataqueues:py:mod:`Signal Display Metrics <ska_sdp_dataqueues.schemas.signal_display_metrics>`.