SKA PST STAT Applications

ska_pst_stat_info

Prints the version of the ska-pst-stat library to stdout.

ska_pst_stat_core

The implementation of the STAT.CORE software component for the Pulsar Timing product. This application regularly (every 5 seconds) processes short (100ms) segments of the voltage data streams received from the Correlator Beam Former. Each segment of data is analysed and a range of statistics metrics are computed. Simple scalar metrics (mean, variance, etc) are provided to a TANGO device and a richer set of vector metrics (bandpasses, histograms, etc) are made available to signal displays. A small HDF5 data file (< 10 MB) is generated from each segment that is processed and is written to the configured output path of <path>/<eb_id>/<subsystem_id>/<scan_id>/monitoring_stats/.

Usage: ska_pst_stat_core

  -c port     port on which to accept control commands
  -d path     write output files to the path [default /tmp]
  -f config   ascii file containing observation configuration
  -h          print this help text
  -t timeout  wait for the specified number of seconds for exiting
  -v          verbose output

ska_pst_stat_file_proc

Computes data quality statistics for every sample in the specified pair of data and weights files, as output by the AA0.5 voltage recorder, producing a single HDF5 output file.

Usage: ska_pst_stat_file_proc -d data -w weights

  -d data     name of data file
  -w weights  name of weights file
  -h          print this help text
  -v          verbose output

The path of the output file is created according to the following rules:

The output filename has the same stem as the data filename with the extension replaced by h5; e.g. if the data filename is something.data then the output filename is something.h5

If the data file is in a parent folder, then the output file is written to the same parent folder with the last sub-folder replaced by stat (if necessary, the stat sub-folder is created); e.g.

if the data filename is parent/something.dada then the output filename is stat/something.h5

if the data filename is /absolute/path/to/something.data then the output filename is /absolute/path/stat/something.h5

plot_stat_file

A python script is provided as an example of how to examine the contents of the HDF5 file that is generated by ska_pst_stat_file_proc. This python script is found in python/src/ska_pst/stat.

usage: plot_stat_file.py [-h] [-c] [-o OUTPUT] stat_filename

positional arguments:
  stat_filename         PST Statistics File in HDF5 format.

optional arguments:
  -h, --help            show this help message and exit
  -c, --contour         plot contoured histogram instead of heat map
  -o OUTPUT, --output OUTPUT
                          save plot to specified output filename

To generate a stats file that can be used with this script, use the ska_pst_generate_file application produced in ska-pst-common and included in the ska-pst builder image.

# Launch an instance of the builder image
make local-dev-env

# Build the release
make local-cpp-build-release

# Generate data and weights files in /tmp/data and /tmp/weights
cd /tmp
testdir=/mnt/ska-pst/src/ska_pst/stat/tests/data
ska_pst_generate_file -d $testdir/data_header_LowAA0.5.txt -w $testdir/weights_header_LowAA0.5.txt -s GaussianNoise -T 0.1

# Generate the HDF5 statistics file
ska_pst_stat_file_proc -d data/2023-02-14-22:15:37_0000000000000000_000000.dada -w weights/2023-02-14-22:15:37_0000000000000000_000000.dada

# Plot the HDF5 statistics using a contour plot for the 2D histogram and output /tmp/stats.png
python /mnt/ska-pst/python/src/ska_pst/stat/plot_stat_file.py -c -o /tmp/stats.png stat/2023-02-14-22:15:37_0000000000000000_000000.h5