visibility_io

Class for orchestrating visibility IO

class ska_sdp_distributed_self_cal_prototype.data_managers.visibility_io.ProcessingSetManager(config: PipelineConfig)[source]

Bases: object

Class for partitioning datasets and distributing disk I/O.

Parameters:: config – Config describing the pipeline.

partitions: List of data partitions. Each partition will be local to a compute node. The data in a partition has uvw locality.

property config: Returns the config attribute.

get_dataset(partition_index: int = 0) → xarray.Dataset[source]

Get measurement dataset from visibility partition.

Parameters:: partition_index – index to visibility partition, default=0.
Returns:: xarray dataset containing measurements.
Return type:: dataset

get_frequencies(partition_index: int = 0)[source]

Get frequencies from visibility partition.

Parameters:: partition_index – index to visibility partition, default=0.
Returns:: Numpy array containing frequency values.

property subgrid_config_list: Returns the subgrid_config_list attribute.

property vis_name: Returns the visibility name.

class ska_sdp_distributed_self_cal_prototype.data_managers.visibility_io.VisibilityPartition(dataset_dict: dict[str, xarray.Dataset] | xradio.correlated_data.ProcessingSet)[source]

Bases: object

A partition of labelled xarray datasets.

dataset_dict: Keys are partition names and values are xarray datasets, where the datasets include updated uvw bounds.

partition_bounds: An xarray DataArray with the maximum and mimimum values of u, v, and w in wavelengths for the whole partition.

sel(**kwargs) → VisibilityPartition[source]

Partitions all datasets by a specific method and condition.

Parameters:

method – the dimension to filter all the datasets by. Options include “time”, “frequency”, “polarization”, or “uvw”.
condition – the filter condition. Can be a single value, a slice, or an array if using “uvw” method.

Returns:

a new object that contains an attribute for the: partitioned datasets.

Return type:

VisibilityPartition

Raises:

ValueError – An error occured applying the method.

total_size_gb() → float[source]

Calculates the total size of all the datasets.

Returns:: Size of the data in the visibility partition in gibibytes.
Return type:: total_gb