RFI simulations

These simulations are designed to provide simulated data containing received signal from input Radio Frequency Interference (RFI) sources. The aim of providing these simulations is to enable testing of RFI-mitigation techniques and to understand the effects of RFI signals at the SKA Mid site.

The following Confluence page provides a detailed summary of how the simulations are generated and the results of testing with these datasets: SKA Mid RFI Simulations

The simulations are generated by running the rfi_sim.sh bash script, which calls the mid_rfi_simulation.py python script. The input data needed for the simulations are stored in ska-sim-mid/scripts/rfi/data.

The bash script has three main parts:

  • First, it runs the mid_rfi_simulation.py script, which takes an HDF5 file as input, and produces a measurement set with RFI signal added to the appropriate frequency channels and time samples.

  • Second, the RASCIL Imager app is called using the measurement set as input and FITS images are generated.

  • Finally, the RASCIL vis_ms visualization app is called to generate various plots from the measurement set.

Input data

The input data need to be supplied in the form of an HDF5 file, with the following information and array structure:

  • Source ID, string, dimensions: (nsources)

  • Source type, string, dimensions: (nsources)

  • Time samples, timestamp, dimensions: (ntimes)

  • Frequency channels, FP64, dimensions: (nfreqs), units: [Hz]

  • SKA station ID, string, dimensions: (nstations)

  • Apparent source coordinates in antenna rest frame, FP64, dimensions: (nsources, ntimes, nants, 3) These are [azimuth, elevation, distance], units: [degree, degree, m]

  • Transmitter power as received by an isotropic antenna, FP64, dimensions: (nsources, ntimes, nants, nfreqs) This does not include the antenna beam pattern which will be applied in the visibility simulation pipeline. units: [W/(Hz m^2)]

Note: at the moment the code is set up to handle Unix time for the provided time samples. This will be updated once the input RFI data of aeronautical sources are generated using MJD [s].

The most up-to-date RFI source data is located in ska-sim-mid/rfi/data/aeronautical_v2_SKA_Mid_coord/aeronautical_sim_datacube_20190313_all.h5. This file contains data for 17 aircraft (RFI sources) for 180 time samples covering one hour of observations, for the full AA1 Mid array (197 antennas) and a single frequency channel (1090 MHz).

ska-sim-mid/rfi/data/aeronautical_v2_SKA_Mid_coord/SKA1MID.csv contains the Mid antenna coordinates used for generating the above data file. The mid RFI simulation code is also updated to generate the telescope configuration using this file. Alternatively, one can use the default RASCIL Mid configuration for the simulations.

Beam gain

The beam gain is applied as part of the RASCIL RFI simulations script simulate_rfi_block_prop, specifically the apply_beam_gain_for_mid function. A voltage pattern for the Mid antennas is generated, together with pointing tables (latter is one for each RFI source). These are used to calculate the gain tables, which are applied to the BlockVisibilities containing the RFI signal, at the end of the simulation.

At the moment we do not have information about the far outside lobes of the Mid dishes, we can only approximate the beam there, which distorts the results further from the beam centre.

We recommend that when using the data file listed above, you do not apply the beam gain directly, or you set the phase centre to be low on the sky, close to the horizon. The aircraft in the data are closer to the horizon, no more than 20 degrees above it. Since the beam we have is small, if the pointing (phase centre) is far from the horizon, all of the RFI signal is missed and removed from the results when the beam gain is applied. If the pointing is set to be closer to the horizon, then some of the aircraft signal will appear in the results (attenuated by the beam as expected).

Python API

Radio Frequency Interference documents the API for two python scripts:

  • the already mentioned mid_rfi_simulation.py, which is the main simulation script

  • and sim_rfi_mid_aa05.py, which can be used to add the simulated RFI signal to an existing MeasurementSet as a noise source.