How to extract data from a saved output file

The event monitor allows you to record the QueryEventSystem() responses received from the monitor Tango devices to a file, but currently does not provide any mechanism to extract this data once it has been recorded. This how-to will guide you through the process of extracting this data.

Iterating snapshots

The file recorded with the --output argument passed to the ska-tango-event-monitor script records each snapshot as a separate JSON object on its own line. These objects can be parsed by iterating over each line of the file and using json.loads:

import json

snapshots = []
with open("events.json") as f:
   for line in f:
      snapshots.append(json.loads(line))

Tip

If using the jq utility, this can be done with the --slurp option.

Note

If using compression, the appropriate compression library must be used to open the file.

Parsing snapshots

Each snapshot has a "time" property that holds a string representation of the timestamp that the snapshot was taken at as well as a "replies" property holding a dictionary mapping admin device names to a response object. Each response object holds either a "data" property, which holds the response to the QueryEventSystem() command from that device server or a "error" property holding a string describing why the response was not collected.

• The "data" property for each device server follows the schema described in QueryEventSystemResponse schema and can be parsed by QueryEventSystem.from_json. For example:

import ska_tango_event_monitor as stem

response = stem.QueryEventSystemResponse.from_json(snapshots[1]["replies"]["dserver/MyServer/1"]["data"])

Generating statistical summaries

If performance monitoring was enabled then the response will have associated lists of ServerPerfSample and/or ClientPerfSample holding performance data. These lists can be summarised using ServerPerfSummary.from_samples or ClientPerfSummary.from_samples. For example:

stats = stem.ServerPerfSummary.from_samples(response.server.perf)