PSRFITS File Format

In PST Detected Filterbank and Pulsar Timing processing modes, output data are written to files in the PSRFITS format, which is compatible with existing Pulsar Timing software such as DSPSR.

FITS and PSRFITS

The Flexible Image Transport System (FITS) (Hanisch et al., A&A, 376, 359, 2001) has been in widespread use for approximately three decades and has evolved into a highly adaptable data storage scheme - see FITS Overview. A wide array of software is available for FITS file manipulation.

PSRFITS (Hotan et al., PASA, 21, 302, 2004) has been developed as a standard FITS-based format for pulsar data files. Both mean pulse profile (“fold-mode”) and streamed (“search-mode”) multi-channel full-polarisation data are supported. PSRFITS is fully integrated with the open-source pulsar data-analysis system PSRCHIVE.

PSRFITS File Structure

In accordance with FITS standards, PSRFITS consists of a primary header-data unit (HDU) followed by a series of extension HDUs. The primary HDU contains basic information such as telescope identification and location, receiver parameters, observation start time, etc. Extension HDUs, formatted as binary tables, contain specific information related to the observation such as the pulsar ephemeris, calibration data and the pulsar data formatted as a series of sub-integrations.

A feature of standard FITS input/output routines is that they are transparent to the addition of new header parameters or even complete HDUs - if they are unknown to the reading program, they are ignored. Furthermore, unused HDUs need not be written, even though they are present in the definition. These features allow, for example, a user group to add information particular to a certain instrument without compromising use of the definition by other groups. A novel feature of the definition is the inclusion of HDUs containing processing history information. For example, the first line of Processing History HDU contains information about the data acquisition program and the initial structure of the file. Subsequent lines record details of modifications to the structure or data, for example, partial or complete dedispersion or interference excision.

HDU Name	Description
PRIMARY	Observer, telescope and receiver information, source name and observation date and time
HISTORY	Date, program and details of data acquisition and each subsequent processing step
OBSDESCR	Free-format ascii description of the observation or signal processing
PSRPARAM	Pulsar ephemeris used to create or modify pulse profile data
POLYCO	History of the TEMPO polyco files used to predict the apparent pulsar period
T2PREDICT	The TEMPO2 predictor file used to predict the apparent pulsar period
COHDDISP	Parameters used for coherent dedisperion of baseband data
BANDPASS	Observed bandpass in each polarisation averaged over observation
FLUX_CAL	System temperature and injected noise calibration data as a function of frequency across the bandpass
CAL_POLN	Apparent polarisation of injected noise calibration signal as a function of frequency
FEEDPAR	Parameters of feed cross-coupling as a function of frequency
SPECKURT	Statistics for spectral kurtosis RFI excision
SUBINT	Pulse profiles or streamed data as a function of time, frequency and polarisation
DIG_STAT	Digitiser mode, attenuator settings and count statistics
DIG_CNTS	Digitiser mode and count rate distribution

When the Detected Filterbank or Pulsar Timing processing modes generate PSRFITS files, they will be split into a sequence of sub-integrations, each containing a configurable length (between 1 and 60 seconds). Each sub-integration will be written to a PSRFITS file that is named with the format:

YYYY-MM-DD:HH:MM:SS.sf

Where the timestamp is the UTC time of the first sample present in the file.

HDU Definitions

The HDUs that will be present in the PSRFITS files are:

• Detected Filterbank : PRIMARY, HISTORY, SUBINT

• Pulsar Timing: PRIMARY, HISTORY, PSRPARAM, T2PREDICT, SUBINT

The definitions of each HDU are described below.

PRIMARY HDU

Key	Value	Units	Description
SIMPLE	T		file does conform to FITS standard
BITPIX	8		number of bits per data pixel
NAXIS	0		number of data axes
EXTEND	T		FITS dataset may contain extensions
HDRVER	6.9		Header version
FITSTYPE	PSRFITS		FITS definition for pulsar data files
DATE			File creation date (YYYY-MM-DDThh:mm:ss UTC)
OBSERVER			Observer name(s)
PROJID			Project name
TELESCOP			Telescope name, Should conform with TEMPO2 usage.
ANT_X		m	Antenna ITRF X-coordinate (D)
ANT_Y		m	Antenna ITRF Y-coordinate (D)
ANT_Z		m	Antenna ITRF Z-coordinate (D)
FRONTEND			Receiver ID, Observatory standard name for receiver package
IBEAM			Beam ID for multi-beam systems, Normally beam 0 is the on-axis beam.
NRCVR	2		Number of receiver polarisation channels
FD_POLN	LIN		LIN or CIRC, Native polarisation of the feed receptors
FD_HAND	1		+/- 1. +1 is LIN:A
FD_SANG	45	deg	FA of E vector for equal sig in A&B (E)
FD_XYPH		deg	Phase of A^* B for injected cal (E)
BACKEND	PST		Backend ID
BECONFIG			Backend configuration file name
BE_PHASE			0/+1/-1 BE cross-phase:0 unknown,+/-1 std/rev
BE_DCC			0/1 BE down conversion conjugation corrected
BE_DELAY		s	Backend propagation delay from digitiser input
TCYCLE		s	On-line cycle time (D)
OBS_MODE			(PSR, CAL, SEARCH)
DATE-OBS		UTC	Date of observation (YYYY-MM-DDThh:mm:ss UTC)
OBSFREQ		MHz	Centre frequency for observation
OBSBW		MHz	Bandwidth for observation
OBSNCHAN			Number of frequency channels (original)
CHAN_DM		cm -3 pc	DM used for on-line dedispersion
PNT_ID			Name or ID for pointing ctr (multi-beam feeds)
SRC_NAME			Source or scan ID
COORD_MD			Coordinate mode (J2000, GALACTIC, ECLIPTIC)
EQUINOX			Equinox of coords (e.g. 2000.0)
RA			Right ascension (hh:mm:ss.ssss)
DEC			Declination (-dd:mm:ss.sss)
BMAJ		deg	Beam major axis length
BMIN		deg	Beam minor axis length
BPA		deg	Beam position angle
STT_CRD1			Start coord 1 (hh:mm:ss.sss or ddd.ddd)
STT_CRD2			Start coord 2 (-dd:mm:ss.sss or -dd.ddd)
TRK_MODE			Track mode (TRACK, SCANGC, SCANLAT)
STP_CRD1			Stop coord 1 (hh:mm:ss.sss or ddd.ddd)
STP_CRD2			Stop coord 2 (-dd:mm:ss.sss or -dd.ddd)
SCANLEN		s	Requested scan length (E)
FD_MODE			Feed track mode - FA, CPA, SPA, TPA
FA_REQ		deg	Feed/Position angle requested (E)
CAL_MODE			Cal mode (OFF, SYNC, EXT1, EXT2)
CAL_FREQ		Hz	Cal modulation frequency (E)
CAL_DCYC			Cal duty cycle (E)
CAL_PHS			Cal phase (wrt start time) (E)
CAL_NPHS			Number of states in cal pulse (I)
STT_IMJD			Start MJD (UTC days) (J - long integer)
STT_SMJD		s	Start time (sec past UTC 00h) (J)
STT_OFFS		s	Start time offset (D)
STT_LST		s	Start LST (D)

HISTORY HDU - History Binary Table Extension

Contains a history of the file processing operations with key parameters following each operation, with one line per operation.

Key	Value	Units	Description
XTENSION	BINTABLE		Processing history
BITPIX	8		N/A
NAXIS	2		2-dimensional binary table
NAXIS1			width of table in bytes
NAXIS2			number of rows
PCOUNT	0		size of special data area
GCOUNT	1		one data group (required keyword)
TFIELDS	23		number of fields per row
EXTNAME	HISTORY		name of this binary table extension

The binary table consists of NAXIS2 rows with the following 23 fields.

Field	Format	Units	Description
DATE_PRO	24A		Processing date and time (UTC)
PROC_CMD	256A		Processing program and command
SCALE	8A		Units (FluxDen/RefFlux/Jansky)
POL_TYPE	8A		Polarisation identifier
NSUB	1J		Number of Sub-Integrations
NPOL	1I		Number of polarisations
NBIN	1I		Nr of bins per product (0 for SEARCH mode)
NBIN_PRD	1I		Nr of bins per period
TBIN	1D	s	Time per bin or sample
CTR_FREQ	1D	MHz	Band centre frequency (weighted)
NCHAN	1J		Number of frequency channels
CHAN_BW	1D	MHz	Channel bandwidth
REF_FREQ	1D	MHz	Reference frequency
DM	1D	CM -3 PC	DM used for dedispersion
RM	1D	RAD M -2	RM used for RM correction
PR_CORR	1I		Projection of receptors onto sky corrected
FD_CORR	1I		Feed basis correction applied
BE_CORR	1I		Backend correction applied
RM_CORR	1I		RM correction applied
DEDISP	1I		Data de-dispersed
DDS_MTHD	32A		Dedispersion method
SC_MTHD	32A		Scattered power correction method
CAL_MTHD	32A		Calibration method
CAL_FILE	256A		Name of gain calibration file
RFI_MTHD	32A		RFI excision method
AUX_RM_C	1I		Auxiliary Faraday rotation corrected flag
DM_MODEL	32A		Auxiliary dispersion model description
AUX_DM_C	1I		Auxiliary dispersion corrected flag

PSRPARAM HDU - Ephemeris Binary Table Extension

The pulsar ephemeris file describes a pulsar system and is used to generate the phase predictor.

Key	Value	Units	Description
XTENSION	BINTABLE		Pulsar ephemeris
BITPIX	8		N/A
NAXIS	2		2-dimensional binary table
NAXIS1			width of table in bytes
NAXIS2			number of rows in table
PCOUNT	0		size of special data area
GCOUNT	1		one data group (required keyword)
TFIELDS	1		Number of fields per row
EXTNAME	PSRPARAM		Name of this binary table extension

The binary table consists of NAXIS2 rows with a single field each

Key	Format	Units	Description
PARAM	128A		ASCII line from the pulsar ephemeris

T2PREDICT HDU - TEMPO2 Predictor Binary Table Extension

Table used by Tempo2 for prediction of pulse phases. The table consists of a two-dimensional (time and frequency) array of Chebyshev basis functions along with header parameters. See the Tempo2 documentation for a description of the predictor file and how to make use of it.

Key	Value	Units	Description
XTENSION	BINTABLE		Tempo2 Predictor
BITPIX	8		N/A
NAXIS	2		2-dimensional binary table
NAXIS1		bytes	width of table
NAXIS2			number of rows in table
PCOUNT	0		size of special data area
GCOUNT	1		one data group (required keyword)
TFIELDS	1		Number of fields per row
EXTNAME	T2PREDICT		Name of this binary table extension

The binary table consists of NAXIS2 rows with a single field each

Key	Format	Units	Description
PREDICT	128A		ASCII line of a Tempo2 predictor file

SUBINT HDU - Data as a function of time, frequency and polarisation

Table containing the observed power spectra, that is spectra after detection or multiplication. Two modes of observation are catered for:

• fold mode: where the data are synchronously folded at the apparent period of a pulsar using a Tempo polyco file or a Tempo2 predictor file and samples are binned in pulse phase

• search mode: where streamed multichannel data are recorded in successive samples.

In both modes, the data can have a single polarisation (normally the sum of two orthogonal polarisations), two orthogonal polarisations or all four polarisation spectra.

In fold mode, data are summed over a sub-integration time and successive sub-integrations are stored in successive rows of the BINTABLE. In search mode, data are grouped in blocks of NSBLK samples and stored in successive rows of the BINTABLE. To avoid excessive overheads, NSBLK is typically 4096.

Fold-mode data are stored as 16-bit signed integers with elements of the data array in polarisation, channel, and phase bin order with the pulse profile bins in contiguous locations. Before conversion to integers, the mean channel power (averaged over bins and polarisations) during the sub-integration is subtracted from the channel data and the residual is scaled so that the values in the DATA array cover the whole available range (-32768 to 32767). The original observed powers are reconstructed using:

Real value = DATA value * DAT_SCL + DAT_OFFS.

Search-mode data may be stored as 1-bit, 2-bit, 4-bit or 8-bit signed or unsigned integers and are written as a byte array. Data digitised with less than 8 bits are packed with earlier samples in higher-order bits of the byte (i.e., “big-endian”). Elements of the data array are in sample, polarisation and channel order with the spectral channels in contiguous locations.

Prior to few-bit digitisation, search-mode sample spectra are generally normalised and given zero mean by forming (S-R)/R, where S is the observed spectrum and R is an estimate of the bandpass or reference spectrum. This effectively does a bandpass calibration and gives an approximately constant rms deviation across the spectrum in order to optimise the few-bit digitisation. These data are normally analysed directly without application of the scale factors and offsets. If required, the reference spectrum may be reconstructed from the DAT_OFFS and DAT_SCL fields of the table. However note that, if channel running means are used to form the reference spectrum, the recorded values are sampled at sub-integration intervals and do do not necessarily represent the exact values used to form the recorded spectra.

When unsigned integers are used to record the truncated data, a zero offset (ZERO_OFFS) is added to the digitised value. Normally ZERO_OFFS = 2 ^{(NBIT - 1)} - 0.5, but for total-intensity multi-bit data, a smaller value may be used to give more headroom. The original observed values are reconstructed using:

Real value = (DATA value - ZERO_OFFS) * DAT_SCL + DAT_OFFS.

The output data for long search-mode observations may be split in time or frequency and recorded in separate files to keep file sizes at manageable values. It is assumed that the data sampling is continuous across the split files.

Key	Default	Description
XTENSION	BINTABLE	Subintegration data
BITPIX	8	N/A
NAXIS	2	2-dimensional binary table
NAXIS1		width of table in bytes
NAXIS2		Number of rows in table (NSUBINT)
PCOUNT	0	size of special data area
GCOUNT	1	one data group (required keyword)
TFIELDS	18	Number of fields per row
EPOCHS		Epoch convention (VALID, MIDTIME, STT_MJD)
INT_TYPE		Time axis (TIME, BINPHSPERI, BINLNGASC, etc)
INT_UNIT		Unit of time axis (SEC, PHS (0-1), DEG)
SCALE		Intensity units (FluxDen
POL_TYPE		Polarisation identifier (e.g., AABBCRCI, AA+BB)
NPOL		Nr of polarisations
TBIN		[s] Time per bin or sample
NBIN		Nr of bins (PSR/CAL mode; else 1)
NBIN_PRD		Nr of bins/pulse period (for gated data)
PHS_OFFS		Phase offset of bin 0 for gated data
NBITS		Nr of bits
ZERO_OFF		Zero offset for SEARCH-mode data
SIGNINT		1 for signed ints in SEARCH-mode data, else 0
NSUBOFFS		Subint offset (Contiguous SEARCH-mode files)
NCHAN		Number of channels
CHAN_BW		[MHz] Channel
REFFREQ		[MHz] Reference frequency
DM		[cm-3 pc] DM for post-detection dedisperion
RM		[rad m-2] RM for post-detection deFaraday
NCHNOFFS		Channel/sub-band offset for split files
NSBLK		Samples/row (SEARCH mode, else 1)
NSTOT		Total number of samples (SEARCH mode, else 1)
NINFO		Number of additional information columns
EXTNAME	SUBINT	name of this binary table extension

The binary table consists of NAXIS2 rows with 18 fields each

Key	Format	Units	Description
INDEXVAL	1D		Optionally used if INT_TYPE !
TSUBINT	1D	s	Length of subintegration
OFFS_SUB	1D	s	Offset from Start of subint centre
LST_SUB	1D	s	LST at subint centre
RA_SUB	1D	deg	RA (J2000) at subint centre
DEC_SUB	1D	deg	Dec (J2000) at subint centre
GLON_SUB	1D	deg	Gal longitude at subint centre
GLAT_SUB	1D	deg	Gal latitude at subint centre
FD_ANG	1E	deg	Feed angle at subint centre
POS_ANG	1E	deg	Position angle of feed at subint centre
PAR_ANG	1E	deg	Parallactic angle at subint centre
TEL_AZ	1E	deg	Telescope azimuth at subint centre
TEL_ZEN	1E	deg	Telescope zenith angle at subint centre
AUX_DM	1D	cm -3 pc	additional DM (ionosphere, corona, etc.)
AUX_RM	1D	rad M -2	additional RM (ionosphere, corona, etc.)
DAT_FREQ	#D	MHz	Centre frequency for each channel, #=NCHAN
DAT_WTS	#E		Weights for each channel, #=NCHAN
DAT_OFFS	#E		Data offset for each channel, #=NCHAN*NPOL
DAT_SCL	#E		Data scale factor (outval=dataval*scl + offs), #=NCHAN*NPOL
DATA	#X	Jy	Subint data table. Fold: #=(NBIN,NCHAN,NPOL) X=I. Search: #=(NCHAN,NPOL,NSBLK*NBITS/8), X=B (1-8 bit)

FITS Type Formats

The Binary Extension tables specify a Format for the binary components that consist of a number (e.g. 1) and the type (e.g. E) that define the number of values of the specified type. When the number is specified as a #, refer to the description for the number of values. The intepretation of the types are described below.

Type	C Type	Description
B	double	Bytes, packed sub-byte integer values (e.g. 1, 2, 4 or 8 bits)
D	double	64-bit floating point value
E	float	32-bit floating point value
A	char	ASCII character
I	int32	Integer: 32-bit signed integer
J	int64	Long: 64-bit signed integer
K	uint64	Long Unsigned: 64-bit unsigned integer
M	std::complex<double>	128-bit complex-valued floating point value
V	uint32	Unsigned: 32-bit unsigned integer
X	N/A	variable data type, refer to the description