SKA PST DSP Applications

ska_pst_dsp_core

The implementation of the DSP.CORE software component for the Pulsar Timing product. This application is used for any of the PST processing modes. This application can be used for the following:

writing beam-formed voltage data to disk as part of the AA0.5 PST Voltage Recorder
writing decimated beam-formed voltage data to disk as part of the AA1 PST Flow Through mode

By default the data streams will be written to sub-directories in /tmp, unless the option is overridden via the -d option. Note that the O_DIRECT option will use raw/direct I/O to the file system to bypass kernel buffering but should only be used on local file systems that support it.

Usage: ska_pst_dsp_core [options] port
Execute DSP CORE application from the configuration file or via control command.
Quit with CTRL+C.

  port        port on which to accept control commands
  -d path     write output files to the recording path [default /tmp]
  -g gpu_id   CUDA capable GPU id
  -h          print this help text
  -n          disable creation of output files, useful for testing on servers with slow file systems
  -o          use O_DIRECT for I/O transactions
  -r          record the time spent in each DSPSR operation
  -v          increase verbosity
  -V          increase verbosity of libDSPSR

ska_pst_dsp_disk

The implementation of the DSP.DISK software component for the Pulsar Timing product. This application is used to write beam-formed voltage data to disk as part of the AA0.5 PST Voltage Recorder. By default the data streams will be written to sub-directories in /tmp, unless the option is overridden via the -d option. Note that the O_DIRECT option will use raw/direct I/O to the file system to bypass kernel buffering but should only be used on local file systems that support it.

This application is deprecated in favour of the ska_pst_dsp_core application. This application can no long be run as long running service.

Usage: ska_pst_dsp_disk [options] config
Create DSP Disk Recorder from the configuration file.
Quit with CTRL+C.

  config      ascii file containing observation configuration
  -d path     write output files to the recording path [default /tmp]
  -h          print this help text
  -n          disable creation of output files, useful for testing on servers with slow file systems
  -o          use O_DIRECT for I/O transactions
  -v          verbose output

ska_pst_dsp_ft

The implementation of the DSP.FT software component for the Pulsar Timing product. This application is used to write decimated beam-formed voltage data to disk as part of the AA1 PST Flow Through mode of operation. By default the data streams will be written to sub-directories in /tmp, unless the option is overridden via the -d option. Note that the O_DIRECT option will use raw/direct I/O to the file system to bypass kernel buffering but should only be used on local file systems that support it.

This application is deprecated in favour of the ska_pst_dsp_core application. This application can no long be run as long running service.

Usage: ska_pst_dsp_core [options] port
Execute DSP CORE application via gRPC service.
Quit with CTRL+C.

  port        port on which to accept control commands
  -d path     write output files to the recording path [default /tmp]
  -g gpu_id   CUDA capable GPU id
  -h          print this help text
  -n          disable creation of output files, useful for testing on servers with slow file systems
  -o          use O_DIRECT for I/O transactions
  -r          record the time spent in each DSPSR operation
  -v          increase verbosity
  -V          increase verbosity of libDSPSR

ska_pst_dsp_disk_perftest

Tests the write performance of the FileWriter class when writing data from a ring buffer to the file system provided in the -d argument. The configuration options support

Usage: ska_pst_dsp_disk_perftest [options] config

  config      ascii file containing observation configuration
  -d path     write files to path [default /tmp]
  -h          print this help text
  -l nsecs    generate data stream for nsecs [default 30]
  -o          use O_DIRECT for file output
  -v          verbose output

ska_pst_dsp_disk_monitor

Test the monitoring functions of the DiskMonitor class when monitoring a file system provided by the -d option.

Usage: ska_pst_dsp_disk_monitor [options] config
Monitor the file system specified in the configuration file and report the disk available statistics. Quit with CTRL+C.

  config      ascii file containing observation configuration
  -d path     recording base path to monitor [default /tmp]
  -h          print this help text
  -v          verbose output

ska_pst_dsp_disk_rand_generate

Generate data and weights data streams, write a pseudo-random sequence to the data and weights shared-memory ring buffers defined in the config file. The configuration file must contain ascii key/value pairs that include the following parameters:

DATA_KEY 4-character hexadecimal key for the PSRDADA ring buffer defining the data stream
WEIGHTS_KEY 4-character hexidemical key for the PSRDADA ring buffer defining the weights stream
NCHAN Number of channels
NBIT Number of bits per sample
NDIM Number of dimensions (1=real, 2=complex)
NSAMP_PP Number of samples per packet/weight
TSAMP Sampling interval in microseconds

Usage: ska_pst_dsp_disk_rand_generate [options] config

config      ascii file containing observation configuration
-h          print this help text
-l nsecs    generate data stream for nsecs [default 15]
-v          verbose output

ska_pst_dsp_disk_rand_validate

Validate a data file written by ska_pst_dsp_disk against a predictable pseudo-random sequence. The unsigned 8-bit integer sequence is seeded using the unix epoch of the UTC_START header attribute, with the position in the sequence determined by the byte-offset from the start of the data stream.

Usage: ska_pst_dsp_disk_rand_validate [options] data_file

data_file   raw data file that will be validated against the pseudo-random sequence
-h          print this help text
-v          verbose output

ska_pst_dsp_disk_sine_analyse

Unpack the timeseries in a data and weights file to produce a denormalised time series. The application will print the frequency channel at which the maximum power is detected. Optionally dump the integrated bandpass to binary file for plotting in matplotlib.

Usage: ska_pst_dsp_disk_sine_analyse [options] data_file weights_file

data_file     raw data file that will be validated against the pseudo-random sequence
weights_file  weights file containing scale and weights corresponding to the data file
-o output     write bandpass to output file raw/binary FP ordered file
-h            print this help text
-v            verbose output

The following python code can plot the contents of the bandpass output file:

import numpy as np
import matplotlib.pyplot as plot

fptr = open("output_file.raw", "rb")
nchan = np.fromfile(fptr, dtype=np.uint32, count=1)[0]
npol = np.fromfile(fptr, dtype=np.uint32, count=1)[0]

fig, axs = plt.subplots(int(npol), 1)
fig.suptitle(f"Bandpass of {nchan} channels with {npol} polarisations")

x_axis = np.fromfile(fptr, dtype=np.float32, count=nchan)
for ipol in range(npol):
    y_axis = np.fromfile(fptr, dtype=np.float32, count=nchan
    axs[ipol].plot(x_axis, y_axis[ipol])
    axs[ipol].set_title(f"Polarisation {ipol}")
plt.xlabel("Frequency [MHz]")
fig.tight_layout()
plt.show()

ska_pst_dsp_disk_data_unpack

Read a data and weights file from disk and write an unpacked file to disk. The unpacked_file will be written in 32-bit floating point PSRDADA format in Time, Frequency, Polarisation ordering, which will allow it to be read by the default FloatUnpacker class of DSPSR.

Usage: ska_pst_dsp_disk_data_unpack [options] data_file weights_file output_dir

data_file     raw data file containing packed data samples
weights_file  weights file containing scale and weights corresponding to the data file
output_dir    directory to write the unpacked file
-h            print this help text
-o            use O_DIRECT for writing file output
-v            verbose output