RFI flagging functions

C/C++

void sdp_flagger_dynamic_threshold(const sdp_Mem *vis, sdp_Mem *flags, const double alpha, const double threshold_magnitudes, const double threshold_variations, const double threshold_broadband, const int sampling_step, const int window, const int window_median_history, sdp_Error *status)

Basic RFI flagger implementing the FluctuFlagger algorithm.

Array dimensions are as follows, from slowest to fastest varying:

  • vis is 2D and complex-valued, with shape:

    • [ num_timesamples, num_channels]

  • thresholds is 1D and real-valued.

    • The size of the array is n, where 2^(n-1) = max_sequence_length .

  • flags is 2D and integer-valued, with the same shape as vis .

Parameters:

  • vis – complex valued visibilities. Dimensions as above.

  • flags – output flags. Dimensions as above.

  • alpha – coefficient for the recursive transit score value

  • threshold_magnitudes – threshold on the modified z-score for magnitudes

  • threshold_variations – threshold on the modified z-score for variations in the magnitudes

  • threshold_broadband – threshold on the modified z-score for median history

  • sampling_step – the intervals at which samples are taken for computing the medians.

  • window – the number of channels on each side of a flagged visibility to be flagged

  • window_median_history – the size of the window of time samples for which the median across all channels is maintained

  • status – Error status.

Python

ska_sdp_func.visibility.flagger_dynamic_threshold(vis: ndarray, flags: ndarray, alpha: float, threshold_magnitudes: float, threshold_variations: float, threshold_broadband: float, sampling_step: int, window: int, window_median_history: int)

A lightweight RFI flagger to statistically flag the unusually larger absolute values of visibilities, the unusually fluctuating absolute values, and unusually large collection of channels with a sudden jump in their absolute values to detect broadband RFI.

Array dimensions are as follows, from slowest to fastest varying:

  • vis is 4D and complex-valued, with shape:

    • [ num_timesamples, num_baselines, num_channels, num_polarisations ]

  • flags is 4D and integer-valued, with the same shape as vis.

Parameters:

  • vis (numpy.ndarray) – Complex valued visibilities. Dimensions as above.

  • flags (numpy.ndarray) – Output flags. Dimensions as above.

  • alpha – The coefficient for the recursive equation for measuring the rate of fluctuations in the ‘recent time samples’, between 0 and 1.

  • threshold_magnitudes – The threshold on modified z-score to flag based on magnitudes of visibilities. A recommended rule of thumb in the statistics textbooks for this value is 3.5.

  • threshold_variations – The threshold on modified z-score to flag based on the rate of fluctuations in the magnitudes in the recent time samples (similar to last one, 3.5 is an appropriate value).

  • threshold_broadband – A threshold on the modified z-score of the medians of each time slot across all channels to detect broadband RFI.

  • sampling_step – Integer value, shows at how many steps we take samples to compute the medians and z-scores across all channels in a time slot.

  • window – The number of side channels of a flagged visibility on each side to be flagged.

  • window_median_history – The window size that the history of the medians of time slots is maintained for the broadband RFI detection.