API Overview

This documentation describes the QA API itself. The main consumer of this API is the QA Display web application, which connects via WebSockets and REST endpoints to present real-time charts and metrics. For instructions on running and integrating the QA Display, see its own documentation.

Getting Started with the API

This documentation assumes you have the API running and accessible, typically at http://localhost:8002.

For local development, use the Swagger Documentation for an interactive API panel. Swagger is also available for any deployed instance.

Note

Swagger documentation does not include WebSocket endpoints. For those, please refer to the source code.

Finding API Endpoints

To explore available endpoints, check the files in: /src/ska_sdp_qa_data_api/server/api

You may also want to review: /src/ska_sdp_qa_data_api/server/api/__init__.py which sets the base paths for each API subsection.

REST Endpoints

These endpoints can all be accessed directly from the browser, and only allow for GET requests. This is only going to be a subset of the endpoints, for more details go to the Swagger Documentation, on any install this can be retrieved by going to /docs.

Ping

This endpoint can be used to determine if the API is responding to requests.

$ curl http://localhost:8002/ping
{"ping":"live"}

Config

This endpoint provides you with details on the current setup. Keys not listed here should be ignored.

$ curl http://localhost:8002/config

{
  "app_version": "1.3.0",
  "project_name": "SDP QA Metrics API",
  "start_time": "2024-09-20T08:12:13.588802+00:00",
  "uptime": 1547.5995180606842
}

Known Subarrays

To be able to get the subarrays, you need to connect to the Configuration Data Websocket. And request from there.

Flows

Flows are the method with which kafka topics are retrieved from configured data flows. The currently setup topics are specified per flow entry. The processing_block_id parameter is optional, and will filter the flow entries as required. It is suggested to always filter this way.

$ curl http://localhost:8002/config/flows?processing_block_id=pb-testrealtime-20240913-95421
[
  {
    "processing_block_id": "pb-testrealtime-20240913-95421",
    "name": "metrics-amplitude-01-pb-testrealtime-20240913-95421",
    "kind": "data-queue",
    "topic": "metrics-amplitude-01-pb-testrealtime-20240913-95421",
    "format": "msgpack_numpy",
    "metric": "amplitude",
    "nchan_avg": 50,
    "rounding_sensitivity": 5,
    "data_model": "MetricPayload"
  },
  ...
  {
    "processing_block_id": "pb-testrealtime-20240913-95421",
    "name": "metrics-uvcoverage-01-pb-testrealtime-20240913-95421",
    "kind": "data-queue",
    "topic": "metrics-uvcoverage-01-pb-testrealtime-20240913-95421",
    "format": "msgpack_numpy",
    "metric": "uvcoverage",
    "nchan_avg": 50,
    "rounding_sensitivity": 5,
    "data_model": "MetricPayload"
  }
]

WebSocket Endpoints

Metric Data

These endpoints should be connected to using a websocket to get data. The format of the data in any websocket call is dependent on the topic, based on the flow configuration.

The API can currently support json and msgpack formats (with msgpack being the most common). Regardless of which format is chosen the output of the first message is always the same, an object with a single key/value pair. That looks like {"status": "connected"}.

Once that is received the websocket will start sending data as is available.

The topics that can be used should be retrieved by using the Flows endpoint.

The URL to get to a websocket is /ws/<topic>. With an optional partition (which defaults to 0). To specify a partition use /ws/<topic>/<partition>.

The URL to get to a websocket pointed at the earliest available message from Kafka is /ws/earliest/<topic>. With an optional partition (default 0). To specify the partition use /ws/earliest/<topic>/<partition>.

Configuration Data

This websocket is a bi-directional websocket, in which data can be requested, either once off or an update can be sent with any change (with an initial state sent). This end point always sends and receives data in the JSON data structure.

This end point is connected to at /ws/ws and the layout of the data that should be sent looks like:

{
    "action": "listen",
    "type": "subarray_state",
    "id": "01"
}

The actions that can be used are get, listen, or config. The config action is special in that it will give the current server stats, but needs to be retrieved whenever you want it updated.

The get and listen can take various type s:

  • list_subarrays - This will list available subarray IDs.

  • subarray_state - This will give the full state (including running execution blocks, and processing blocks) of the given subarray, the subarray ID should be given in the id key.

  • processing_block - This will give the configuration of a processing block, the processing block ID should be given in the id key.

  • list_flows - This will list all available flow entries.

  • flow - Retrieve a single flow entry. But Flow ID.

In most cases the most used actions should be listen being executed on list_subarrays and subarray_state, the others should rarely be used.