Data products of the pointing pipeline

The pointing pipeline outputs several files, written to disc, for further analysis. These data products are described here.

HDF5 main data product

The main data product of the pointing pipeline is an HDF5 file based on the PointingTable data model. One file per pointing observation is generated. The file is set up to provide all relevant information for individual reference pointing observations, as well as the inputs required to generate an offline global pointing model. For data with 61 dishes and 4096 frequency channels, the HDF5 output file for a single reference pointing observation is 28kB in size.

HDF5 format is a self describing data format containing metadata (attributes) and binary data organised in a “file directory” like structure. The detailed format of the pointing output product is shown in the tables below. Each file contains the results from one pointing observation - i.e. a single PointingTable object in the HDF5 group labelled "PointingTable0".

Product quality

The HDF5 file contains the fitted offsets and their uncertainties. The fitted offsets must pass a number of quality criteria (see Validity of the Gaussian Fits) in order to be written into the output. If these criteria are not met, the output offsets are set to NaN.

Accessing and reading the files

Details of how to locate and read the HDF5 files are described in a Jupyter notebook.

Global attributes

Attribute Name	Type	Description
number_data_models	long	Number of data models contained in file

Group attributes

Attribute Name	Type	Description
data_model	string	Python data model class, i.e. `PointingTable`
pointing_frame	string	Frame of reference used for pointing fits, e.g. `"xel-el"`
pointingcentre_coords	string	Pointing centre RA/Dec coordinates in degrees
pointingcentre_frame	string	Coordinate frame for pointing centre, e.g. `"icrs"`
receptor_frame	string	Polarisation state, e.g. `"StokesI"`
band_type	string	Observing band, e.g. `"Band 2"`
scan_mode	string	Pointing observation mode, e.g. `"5-point"`
track_duration	double	How long each scan position was tracked for in seconds. Multiple pointing timestamps are required to calculate the integration time and hence the `track_duration`. If only one pointing timestamp is present, `None` is stored as the track duration
discrete_offset	double	Discrete offset of each scan from the central scan, in degrees (xel-el)
commanded_pointing	double	Commanded pointing of each dish at the middle timestamp of the central scan of the pointing observation (radians)

Group data

Dimensions:

nTime: number of time chunks (currently=1)
nAnt: number of antennas
nFreq: number of frequency chunks (currently=1)
nRecep: number of polarisations (currently=1, “StokesI”)
nCoord: number of coordinate angles in pointing frame - i.e. 2 for (xel, el)

Name	Type (dimensions)	Unit	Description
configuration	sub-group		Data model for telescope configuration containing antenna characteristics following the MSv3 definition (see below for full description).
data_expected_width	double (nTime, nAnt, nFreq, nRecep, nCoord)	rad	The theoretical voltage beam sizes for all antennas in radians in the horizontal and vertical co-polarisations
data_fitted_height	double (nTime, nAnt, nFreq, nRecep)		The fitted Gaussian height for all antennas in arbitrary units
data_fitted_height_std	double (nTime, nAnt, nFreq, nRecep)		The standard error on the fitted_height in arbitrary units
data_fitted_width	double (nTime, nAnt, nFreq, nRecep, nCoord)	rad	The fitted voltage beam sizes for all antennas in the horizontal and vertical co-polarisations
data_fitted_width_std	double (nTime, nAnt, nFreq, nRecep, nCoord)	rad	The standard error on the fitted_width
data_frequency	double (nFreq)	Hz	The central frequency in Hz if fitting to gains and frequency at the higher end of the band if fitting to visibilities
data_pointing	double (nTime, nAnt, nFreq, nRecep, nCoord)	rad	The pointing offsets in cross-elevation and elevation in radians for all antennas
data_time	double (nTime)	s	The time in MJD at the middle of all the scans
data_weight	double (nTime, nAnt, nFreq, nRecep, nCoord)	rad^-2	The inverse square of the standard error in the fitted pointing values

Configuration sub-group

Data model for telescope configuration containing antenna characteristics following the MSv3 definition. Note: the sub-group data are accessed via nested sub-subgroup also named “configuration”.

Configuration sub-group attributes
Attribute Name	Type	Description
data_model	string	Python data model class, i.e. `Configuration`
name	string	Telescope name
location	string	xyz location in m
frame	string	Reference coordinate frame for location
receptor_frame	string	Polarisation state, e.g. “StokesI”

Configuration sub-group data
Name	Type (dimensions)	Unit	Description
diameter	double (nAnt)	m	Dish diameter
mount	string (nAnt)		Mount type of the antenna eg. “azel”
names	string (nAnt)		Dish names
offset	double (nAnt, 3)	m	Axes oﬀset of mount to feed reference point
stations	string (nAnt)		Station names
vp_type	string (nAnt)		Voltage pattern type eg “MID”
xyz	double (nAnt, 3)	m	Antenna positions in Earth-Centred Earth Fixed coordinates

Metadata yaml file

Each HDF5 file has a corresponding yaml file that contains additional metadata items. The yaml metadata file contains:

Execution block ID
The context of the execution block provided by the Observation Execution Tool (OET)
The configuration of the processing block used to generate the data products
A list of data product files with description, status, size and a cyclic redundancy check (CRC) for error detection
Associated IVOA ObsCore attributes used for querying astronomical observations

The IVOA ObsCore attributes currently implemented for pointing files are listed in the table below.

Metadata name	Type	Unit	Description
dataproduct_type	String		Data type
calib_level	Integer		Calibration level (0, 1, 2, 3, 4)
obs_collection	String		Name of the data collection
obs_id	String		Observation ID
access_format	String		File content format
access_estsize	integer	kB	Estimated size of dataset
target_name	String		Astronomical object observed, if any
s_ra	double	deg	Central right ascension, ICRS
s_dec	double	deg	Central declination, ICRS
t_min	double	s	Start time in MJD
t_max	double	s	Stop time in MJD
em_min	double	m	Start wavelength
em_max	double	m	Stop wavelength
pol_states	String		List of polarization states or NULL if not applicable
pol_xel	integer		Number of polarization samples
facility_name	String		Name of the facility used for this observation
instrument_name	String		Name of the instrument used for this observation

Additional outputs

TXT file

A CSV-type text file is generated with the following header:

AntennaName,
CrossElevationOffset, CrossElevationOffsetStd,
ElevationOffset, ElevationOffsetStd,
ExpectedWidthH, ExpectedWidthV,
FittedWidthH, FittedWidthHStd,
FittedWidthV, FittedWidthVStd,
FittedHeight, FittedHeightStd

For the description of each column, see the corresponding data item in the HDF5 file description above. However, note that contrary to the HDF5 file, the cross-elevation and elevation offsets have their standard deviations listed in the text file, and not the weight values. H stands for horizontal, V stands for vertical co-polarisation.

PNG files

The pipeline (when run in standalone mode) produces various plots for validation purposes.

Visibility amplitudes: The visibility amplitudes for the parallel-hands polarisations when fitting the beams to the visibility amplitudes option is used
Gain amplitudes: The un-normalised gain amplitudes for all antennas across the entire frequency band or sub-bands when fitting the beams to the gain amplitudes option is used
On-sky offsets: The commanded pointings relative to the source in cross-elevation and elevation in the reference pointing observation
Gaussian fits: The Gaussian fit in cross-elevation and elevation showing the fitted centre across the entire frequency band or in all sub-bands
Pointing offsets: The weighted average of the pointing offsets shown in the Gaussian fits, other fitted parameters, and their uncertainties