Freq Sweep Check

Frequency sweep observation report generation and spreadsheet utilities.

ska_sci_ops_data_analysis.freq_sweep_check.append_calsolsummarydata_to_buildobsreport(freqsweeps: DataFrame, path_calsolsummaries: str)

Augment frequency sweep data with calibration solution summaries.

This function merges or enriches the provided frequency sweep records with calibration solution summary information from a CSV or similar source.

Parameters:

  • freqsweeps (Any) – Data structure containing frequency sweep results (e.g., DataFrame or list of observations).

  • path_calsolsummaries (str) – Path to the calibration solution summaries file.

Returns:

Updated frequency sweep data with calibration summary information merged or appended.

Return type:

Any

ska_sci_ops_data_analysis.freq_sweep_check.append_notes_antennaflagging(freqsweeps: DataFrame, antenna_flag_notes: DataFrame)

Augment frequency sweep data with notes on antenna flagging.

Parameters:

  • freqsweeps (pd.DataFrame) – Data structure containing frequency sweep results

  • antenna_flag_notes (pd.DataFrame) – Data structure containing antenna_flagging_notes

Returns:

Updated frequency sweep data with notes on highly flagged devices included.

Return type:

Dataframe

ska_sci_ops_data_analysis.freq_sweep_check.build_and_save_observation_report(start_date: str, end_date: str, *, operator: str = 'auto', data_path: str = '/home/jovyan/shared/test_executions/FrequencySweepMultiple/', root_path: str = '/home/jovyan/shared/test_executions/RunCalibrationSFTMultiple', save_dir: str | ~pathlib.Path = '/home/jovyan/shared/auto_freq_sweep_check', excel_prefix: str = 'useful_operations', excel_suffix: str = '1', write_excel: bool = True, show_display: bool = True, path_calsolsummaries: str = '/home/jovyan/shared/calibration_solution_summaries/solns.csv', google_sheet_id: str = '1CK28zswmvDJVgt2HuJvwO7N8Bn53CTg7CBOIgeO8nbw', google_sheet_gid: str | int | None = '1463323547', add_suggested_tickets: bool = True, show_ticket_entry_form: bool = True, ticket_entry_max_rows: int | None = 200, ticket_entry_only_with_suggestions: bool = True, populate_existing_eb_blocks_in_sheet: bool = False, populate_existing_eb_blocks_only_if_blank: bool = True, highlight_existing_eb_block_rows_in_sheet: bool = True, antenna_flagging_notes: ~pandas.core.frame.DataFrame = Empty DataFrame Columns: [] Index: []) → tuple[DataFrame, Path | None]

Build the observation report for a date range and optionally save it.

The report combines frequency-sweep observation results, calibration status, ticket suggestions, and calibration-solution summary data for the requested date range.

Parameters:

  • start_date (str) – Start date of the reporting window in YYYY-MM-DD format.

  • end_date (str) – End date of the reporting window in YYYY-MM-DD format.

  • operator (str) – Which observation rows to include. Supported values are "auto", "manual", and "both".

  • data_path (str) – Path to the root directory containing FrequencySweepMultiple observation folders.

  • root_path (str) – Path to the root directory containing RunCalibrationSFTMultiple calibration folders.

  • save_dir (str | Path) – Directory where the Excel report should be written when write_excel is True.

  • excel_prefix (str) – Prefix used when constructing the Excel output filename.

  • excel_suffix (str) – Suffix used when constructing the Excel output filename.

  • write_excel (bool) – Whether to write the generated report to an Excel file.

  • show_display (bool) – Whether display output should be enabled for notebook use. This parameter is currently accepted for compatibility with existing call sites.

  • path_calsolsummaries (str) – Path to the CSV file containing calibration solution summary data that will be merged into the report.

  • google_sheet_id (str) – Google Sheets spreadsheet ID used to look up ticket history for suggested ticket values.

  • google_sheet_gid (str | int | None) – Google Sheets worksheet GID used to look up ticket history for suggested ticket values.

  • add_suggested_tickets (bool) – Whether to populate the suggested tickets column using historical spreadsheet note-to-ticket matches.

  • show_ticket_entry_form (bool) – Whether to display the notebook preview and posting form for manual ticket entry and spreadsheet posting.

  • ticket_entry_max_rows (int | None) – Maximum number of rows to show in the notebook preview/posting form. Use None to show all rows.

  • ticket_entry_only_with_suggestions (bool) – Whether the ticket-entry workflow should be limited to rows that have suggested tickets. This parameter is currently retained for API compatibility.

  • populate_existing_eb_blocks_in_sheet (bool) – Whether to populate the eb blocks column for rows that already exist in the spreadsheet by matching on date and station name.

  • populate_existing_eb_blocks_only_if_blank (bool) – Whether existing spreadsheet EB block cells should only be populated when blank.

  • highlight_existing_eb_block_rows_in_sheet (bool) – Whether to visually highlight rows in the spreadsheet that already contain EB block information.

  • antenna_flagging_notes (pd.DataFrame) – dataframe with information about devices that were highly flagged in observations, add to notes column.

Returns:

A tuple containing the generated report DataFrame and the path to the written Excel file. The path element is None when no Excel file is written.

Return type:

tuple[pd.DataFrame, Optional[Path]]

Raises:

  • ValueError – If operator is not one of the supported values.

  • AssertionError – If data_path or root_path does not exist.

ska_sci_ops_data_analysis.freq_sweep_check.check_smartboxes(flagged_ants: str)

Check if flagged antennas share a smartbox.

Parameters:

flagged_ants (str) – List of flagged antennas.

Returns:

True if any smartbox has >=10 antennas.

Return type:

bool

ska_sci_ops_data_analysis.freq_sweep_check.compute_qual2(row: Series)

Assign a single reason to column ‘quality 2’ based on priority.

Parameters:

row (pd.Series) – DataFrame row containing observation data.

Returns:

Quality classification string.

Return type:

str

ska_sci_ops_data_analysis.freq_sweep_check.parse_flagged(x: str)

Process or transform a single value.

Parameters:

x (str) – Input value to be processed.

Returns:

List of parsed elements.

Return type:

list[str]