Flow Through Data Products

In the Flow Through processing mode, data are written to disk in PSRDADA files as described at PSRDADA File Format. Three types of files are produced by Flow Through processing mode:

  1. Data Files: Sequence of PSRDADA files containing re-quantised, channelised, voltage time-series.

  2. Weights Files: Sequence of PSRDADA files containing the relative weights describing upstream flagging of the data.

  3. Scales and Offsets: Single PSRDADA file containing scales and offsets required to denormalise the data.

Flow Through Data Files

In each data file produced by Flow Through mode, samples are organized in time, frequency, polarization (or TFP) order, such that time is the mostly slowly changing index and polarization is the most quickly changing. If data are encoded with fewer than eight bits per sample, then TFP order persists from the least significant bit (LSB) to the most significant bit (MSB). For example, if NBIT=2, each byte will contain

LSB Re[p0] Im[p0] Re[p1] Im[p1] MSB

where p0 and p1 are polarisation 0 and 1, and Re[z] and Re[z] are the real and imaginary parts of complex-valued z. Each value is encoded using twos complement.

Flow Through Weights Files

The weights files contain the relative weights that indicate the data quality of the samples to which they correspond. These weights are initially derived from the relative-weights provided by the beam-former in the CBF/PSR packet stream (see Voltage Recorder Data Products for more information).

During Flow Through Processing the weights are organized into time, frequency (or TF) order such that time is the slowest changing index and frequency is the fastest changing. Note that should a sub-set of the available frequency channels be specified in the Flow Through Processing, only the configured channels will be recorded. The 16-bit values for the weights are reinterpreted from a fixed point decimal to unsigned sixteen bit integers, as shown below.

Interpretation of Relative Weights Values

CBFPSR Hex Value

CBF Interpretation

Flow Through Interpretation

0x0000

0

0

0x0001

0.000030517578125

1

0x7FFF

0.999969482421875

32767

0x8000

1.0

32768

0xFFFF

1.999969482421870

65535

Flow Through Scales and Offsets File

During the Flow Through signal processing when the samples are re-quantised from 32-bit floating point precision to the specified output bits/sample, scale and offset factors will be computed and applied to each channel such that available bits are used optimally. These factors are also recorded to an output data file so that the absolute flux scale is preserved.

This file will be named <TIMESTAMP>.scloffs where TIMESTAMP is the UTC timestamp of the first integer second of the observation, written in the format: YYYY-MM-DD-HH:MM:SS.

The scloffs file contains an ASCII header, similar to the data and weights files, with the data portion of the file containing records of binary encoded values:

  1. sample_offset (uint64_t)

  2. packed scales and offsets: (nchan * npol * 2 * float)

Different rescaling strategies in the Flow Through Processing will result in one or more records of scales and offsets. The packed scales and offsets are organized in FP ordering with channel changing slowest, then polarisation and finally scale and offset. A 3 channel, 2 polarisation record is encoded as:

  • sample_offset (uint64)

  • Ch0 Pol0 Scale (float)

  • Ch0 Pol0 Offset (float)

  • Ch0 Pol1 Scale (float)

  • Ch0 Pol1 Offset (float)

  • Ch1 Pol0 Scale (float)

  • Ch1 Pol0 Offset (float)

  • Ch1 Pol1 Scale (float)

  • Ch1 Pol1 Offset (float)

  • Ch2 Pol0 Scale (float)

  • Ch2 Pol0 Offset (float)

  • Ch2 Pol1 Scale (float)