Calibration Solution Set Summary

Class and functions for summarising calibration solution sets.

class ska_sci_ops_data_analysis.cal_summary.SolutionSetSummary(soln_dir: str = None, csv_row: str = None, force_fit: bool = False, force_zero_phase_offset: bool = False)

Summary of a set of calibration solutions.

calc_lst()

Calculate LST of this solution set.

calc_quality()

Determine quality of this calibration solution set.

check_smartboxes()

Check if any smartboxes are entirely/almost entirely flagged.

csvstr()

Generate comma-separated representation of attributes.

Returns:

csv representation of attributes

Return type:

str

estimate_sensitivity_loss(mode='tied-array')

Estimate the sensitivity loss due to quality of solutions.

Parameters:

mode (str, optional) – Station operation mode. Must be one of [“tied-array”, “incoherent”], defaults to “tied-array”

Returns:

Estimated sensitivity loss as a fraction of perfect/no-loss sensitivity

Return type:

float

plot_fmax_phase_err_hists(axs: list = None, bins: ndarray = array([0., 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1., 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2., 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3., 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 4., 4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, 4.8, 4.9, 5., 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7, 5.8, 5.9, 6., 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8, 6.9, 7., 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 8., 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8, 8.9, 9., 9.1, 9.2, 9.3, 9.4, 9.5, 9.6, 9.7, 9.8, 9.9]))

Plot histograms of fmax_phase_errs in both polarisations.

Parameters:

  • axs (list) – Optional list containing two axes to plot on. If provided, axs[0] will be used for X, axs[1] for Y. Defaults to None

  • bins (np.ndarray) – Bins for histograms, defaults to np.arange(0, 10, 0.1)

Returns:

fig, axs

Return type:

plt.Figure, plt.Axes

plot_phase_delta(sep_pols: bool = False, axs: list = None)

Plot the delta between gain phases and their fits.

Useful for checking for systematic errors in the phase gradient fit - a good set of fits should have the deltas around 0 across the whole band.

Parameters:

  • sep_pols (bool) – If True, plots X and Y polarisations on separate axes. Optional, defaults to False.

  • axs (list) – Optional list containing two axes to plot on. If provided, axs[0] will be used for X, axs[1] for Y. Overrides sep_pols (provide same axis twice to plot on one axis). Defaults to None

Returns:

fig, ax. If sep_pols len(ax) == 2, else 1

Return type:

plt.Figure, plt.Axes

Raises:

FileNotFoundError – If gains cannot be found

set_fit_params(force_fit: bool = False, force_zero_phase_offset: bool = False)

Load or calculate phase gradient fit parameters.

Parameters:

  • force_fit (bool) – If True, do not load parameters even if they are available. Defaults to False.

  • force_zero_phase_offset (bool) – If True and fitting is being done, force the fit on the phase gradients to set the offset to zero. Defaults to False.

set_flagged_antennas()

Get list of flagged antennas.

summarise(plot_all_ants=False)

Summarise the quality of a set of calibration solutions.

Parameters:

plot_all_ants (boolan, optional, defaults to False) – if True, plots the solutions and fits for all antennas. WARNING: very slow

Raises:

FileNotFoundError – If gain solutions cannot be found

ska_sci_ops_data_analysis.cal_summary.array_to_str(arr)

Generate a string from an array.

Format will be:

[arr[0] arr[1] … arr[-1]]

Parameters:

arr – array to generate string from

Returns:

string with above format

ska_sci_ops_data_analysis.cal_summary.find_soln_dirs(cal_soln_basedir: str = '/home/jovyan/shared/test_executions/RunCalibrationSFTMultiple', station_names: list = ['s8-1', 's8-2', 's8-3', 's8-4', 's8-5', 's8-6', 's9-2', 's9-3', 's9-4', 's9-5', 's10-1', 's10-2', 's10-3', 's10-5'])

Find calibration solution directories.

Parameters:

  • cal_soln_basedir (str) – Base directory of soluctions, defaults to CAL_SOLN_BASEDIR

  • station_names (list) – list of stations to search for, defaults to STATION_NAMES

Returns:

sorted list of solution directories

Return type:

list

ska_sci_ops_data_analysis.cal_summary.flagged_vs_sigmaphi_scatter(summaries: DataFrame, ax: axis)

Plot summaries in n_flagged_ants - mean_fmax_phase_err space.

Parameters:

  • summaries (pd.DataFrame) – DataFrame of summaries

  • ax (plt.axis) – Axis to plot on

ska_sci_ops_data_analysis.cal_summary.get_date_mask(summaries: DataFrame, start_date: str, end_date: str)

Create a mask for summaries between the dates provided (inclusive).

Parameters:

  • summaries (pd.dataframe) – Dataframe containing summaries loaded from csv

  • start_date (str) – ISO format (YYYY-MM-DD) start date.

  • end_date (str) – ISO format (YYYY-MM-DD) end date.

Returns:

boolean mask of summaries within dates

Return type:

np.ndarray

ska_sci_ops_data_analysis.cal_summary.get_soln_model(soln_dir: str)

Get the model used for a given set of calibration solutions.

Parameters:

soln_dir (str) – Calibration solution set

Returns:

Name of model used

Return type:

str

ska_sci_ops_data_analysis.cal_summary.load_soln_summaries(cal_soln_basedir: str = '/home/jovyan/shared/test_executions/RunCalibrationSFTMultiple', station_names: list = ['s8-1', 's8-2', 's8-3', 's8-4', 's8-5', 's8-6', 's9-2', 's9-3', 's9-4', 's9-5', 's10-1', 's10-2', 's10-3', 's10-5'], csv_file: str = None, csv_and_search: bool = True, force_fit: bool = False, force_zero_phase_offset: bool = False)

Load SolutionSetSummaries from solutions or csv file.

Parameters:

  • cal_soln_basedir (str) – Base directory for solutions, defaults to CAL_SOLN_BASEDIR

  • station_names (list) – Station names to search for if loading from solutions, defaults to STATION_NAMES

  • csv_file (str) – If provided, loads summaries from this file, defaults to None

  • csv_and_search (bool) – If True, will load summaries from csv_file and search for more solutions in cal_soln_basedir that aren’t already in csv_file, defaults to True

  • force_fit – If True, will fit phases even if they already exist in solution directories.

  • force_fit – bool

  • force_zero_phase_offset (bool) – If True and fitting is being done, force the fit on the phase gradients to set the offset to zero. Defaults to False.

Returns:

list of SolutionSetSummaries

Return type:

list

ska_sci_ops_data_analysis.cal_summary.loss_line(x, loss)

Calculate N as function of mean_fmax_phase_err corresponding to loss of l.

Parameters:

  • x – mean_fmax_phase_err

  • loss – sensitivity loss fraction

Returns:

number of flagged antennas for the given x that produces loss.

ska_sci_ops_data_analysis.cal_summary.quality_pie(summaries: DataFrame, ax: axis)

Plot pie chart of summary qualities.

Parameters:

  • summaries (pd.DataFrame) – DataFrame of summaries

  • ax (plt.axis) – Axis to plot on

ska_sci_ops_data_analysis.cal_summary.soln_dir_to_datetime(dir_name)

Generate a datetime object from a solution directory path.

TODO: this should probably go in a different file

Parameters:

dir_name (string) – solution directory path to process

Returns:

datetime object of the time of the solutinos

Return type:

datetime.datetime

ska_sci_ops_data_analysis.cal_summary.summarise_cals_in_date_range(start_date: str, end_date: str, csv_file: str = '/home/jovyan/shared/calibration_solution_summaries/solns.csv', csv_and_search: bool = True)

Produce summary plots for all existing solutions in a time period.

Parameters:

  • start_date (str) – First date to include

  • end_date (str) – Last date to include

  • csv_file (str) – csv file where solution summaries are stored. Optional, defaults to CAL_SUMMARY_CSVFILE

  • csv_and_search (bool) – If True, will load summaries from csv_file and search for more solutions in cal_soln_basedir that aren’t already in csv_file, defaults to True

ska_sci_ops_data_analysis.cal_summary.write_soln_summaries(summaries, csv_file)

Write SolutionSetSummaries to csv file.

Parameters:

  • summaries (list) – List of SolutionSetSummaries to write

  • csv_file (str) – filename to write to