API

ska_sdp_exec_swiftly.api Module

Application Programming Interface for Distributed Fourier Transform

Functions

`make_full_facet_cover`(swiftlyconfig)	make facet config list
`make_full_subgrid_cover`(swiftlyconfig)	make subgrid config list

Classes

`FacetConfig`(off0, off1[, mask0, mask1])	Facet Configuration Class
`SubgridConfig`(off0, off1[, mask0, mask1])	Subgrid Configuration Class
`SwiftlyConfig`(**fundamental_constants)	Swiftly configuration
`SwiftlyForward`(swiftly_config, facet_tasks)	Swiftly Forward class
`SwiftlyBackward`(swiftly_config, ...[, ...])	Swiftly Backward class

ska_sdp_exec_swiftly.api_helper Module

Some helper function for Distributed Fourier Transform API

Functions

`accumulate_column`(distributedFFT, NAF_NAF, ...)	update NAF_MNAF
`accumulate_facet`(distributedFFT, NAF_MNAF, ...)	update MNAF_BMNAF
`check_facet`(N, facet_off0, facet_B0, ...)	check facet using sources
`check_residual`(residual_facet)	check residual image
`check_subgrid`(N, sg_off0, sg_A0, sg_off1, ...)	check subgrid using sources
`extract_column`(distriFFT, BF_F, Fn_task, ...)	extract column task
`finish_facet`(distriFFT, MNAF_BMNAF, Fb, ...)	wrapper of finish_facet
`make_facet`(N, yB_size, facet_off0, facet_B0, ...)	make facet
`make_facet_from_sources`(sources, image_size, ...)	Generates a facet from a source list
`make_full_cover_config`(N, chunk_size, class_name)	Generate a fully covered config list
`make_mask_from_slice`(slice_list, mask_size)	make mask from sparse_mask result
`make_subgrid`(N, xA_size, sg_off0, sg_A0, ...)	make subgrid
`make_subgrid_from_sources`(sources, ...[, ...])	Generates a subgrid from a source list
`prepare_and_split_subgrid`(distributedFFT, ...)	prepare NAF_NAF
`sum_and_finish_subgrid`(distributedFFT, ...)	sum faect contribution and finsh subgrid

ska_sdp_exec_swiftly.fourier_transform.algorithm_parameters Module

The main data classes are listed in this module.

Note: An alternative way of calculating nfacet is:

if sizes.fov is not None:
    nfacet = int(numpy.ceil(sizes.N * sizes.fov / sizes.yB_size))
    log.info(
        f"{nfacet}x{nfacet} facets for FoV of {sizes.fov} "
        f"({sizes.N * sizes.fov / nfacet / sizes.yB_size * 100}% efficiency)"
    )

This makes sure that if we have a specific FoV we care about, then we don’t create facets outside that.

Classes

`BaseArrays`(**fundamental_constants)	Class that calculates and holds fundamental constant arrays.
`BaseParameters`(**fundamental_constants)	**fundamental_constants contains the following keys:
`StreamingDistributedFFT`(**fundamental_constants)	Streaming Distributed Fourier Transform class

ska_sdp_exec_swiftly.fourier_transform.fourier_algorithm Module

Distributed Fourier Transform Module. Included are a list of base functions that are used across the code.

Functions

`broadcast`(a, dims, axis)	Stretch input array to shape determined by the dims and axis values.
`coordinates`(n)	Generate a 1D array with length n, which spans [-0.5,0.5] with 0 at position n/2.
`create_slice`(fill_val, axis_val, dims, axis)	Create a tuple of length = dims.
`dask_wrapper`(func)	The Dask wrapper function
`extract_mid`(a, n, axis)	Extract a section from middle of a map (array) along a given axis.
`fft`(a, axis)	Fourier transformation from image to grid space, along a given axis.
`ifft`(a, axis)	Fourier transformation from grid to image space, along a given axis.
`ith_subgrid_facet_element`(true_image, ...[, ...])	Calculate a single facet or subgrid element.
`make_facet_from_sources`(sources, image_size, ...)	Generates a facet from a source list
`make_subgrid_and_facet`(G, FG, base_arrays, dims)	Calculate the actual subgrids and facets.
`make_subgrid_and_facet_from_hdf5`(G, FG, ...)	Calculate the actual subgrids and facets.
`make_subgrid_and_facet_from_sources`(sources, ...)	Calculate the actual subgrids and facets from a list of specific sources.
`make_subgrid_from_sources`(sources, ...[, ...])	Generates a subgrid from a source list
`pad_mid`(a, n, axis)	Pad an array to a desired size with zeros at a given axis.
`roll_and_extract_mid`(shape, offset, ...)	Calculate the slice of the roll + extract mid method
`roll_and_extract_mid_axis`(data, offset, ...)	Calculate the slice of the roll + extract mid along

ska_sdp_exec_swiftly.dask_wrapper Module

The dask wrapper.

We wrap the Dask delayed implementation in a dask wrapper where if use_dask is set to True, the dask wrapper function will call the dask.delayed option for the computation.

Functions

`dask_wrapper`(func)	The Dask wrapper function
`set_up_dask`([scheduler_address])	Set up the Dask Client
`tear_down_dask`(client)	Close the Dask Client

ska_sdp_exec_swiftly.utils Module

Utility Functions

We provide functions that help plotting and basic validation of the algorithm.

Functions

`add_two`(one, two, **kwargs)	Functions for iterative operations to accelerate the construction of graphs
`coordinates`(n)	Generate a 1D array with length n, which spans [-0.5,0.5] with 0 at position n/2.
`dask_wrapper`(func)	The Dask wrapper function
`display_plots`(x[, legend, grid, xlim, fig_name])	Display plots using pylab
`error_task_facet_to_subgrid_2d`(approx, ...)	Calculate the error terms for a single facet to subgrid algorithm.
`error_task_subgrid_to_facet_2d`(approx, ...)	Calculate the error terms for a single 2D subgrid to facet algorithm.
`errors_facet_to_subgrid_2d`(NMBF_NMBF, ...[, ...])	Calculate the error terms for the 2D facet to subgrid algorithm.
`errors_facet_to_subgrid_2d_dask`(...)	Functions for calculating the error of approx subgrid
`errors_subgrid_to_facet_2d`(BMNAF_BMNAF, ...)	Calculate the error terms for the 2D subgrid to facet algorithm.
`errors_subgrid_to_facet_2d_dask`(...)	Functions for calculating the error of approx facets
`extract_mid`(a, n, axis)	Extract a section from middle of a map (array) along a given axis.
`fft`(a, axis)	Fourier transformation from image to grid space, along a given axis.
`fundamental_errors`(approx_what, number_what, ...)	Functions for calculating the error common to facet or subgrid
`generate_input_data`(sparse_ft_class[, ...])	Generate standard data G and FG
`ifft`(a, axis)	Fourier transformation from grid to image space, along a given axis.
`mark_range`(lbl, x0[, x1, y0, y1, ax, ...])	Helper for marking ranges in a graph.
`mean_img_task`(err_img, **kwargs)	the mean of err_mean, err_mean_img
`pad_mid`(a, n, axis)	Pad an array to a desired size with zeros at a given axis.
`plot_pswf`(constants_class[, fig_name])	Plot to check that PSWF indeed satisfies intended bounds.
`plot_work_terms`(constants_class[, fig_name])	Calculate actual work terms to use and plot to check them.
`roll_and_extract_mid`(shape, offset, ...)	Calculate the slice of the roll + extract mid method
`single_write_hdf5_task`(hdf5_path, ...)	Single subgrid or facet write hdf5 file task
`sum_error_task`(err_list, **kwargs)	Summing over error array :param err_list: err_mean, err_mean_img list from subgrid or facets :returns: sum of err_mean and err_mean_img # TODO: need reduce sum
`trim`(ls, **kwargs)	Fetch the first element of a list
`write_hdf5`(approx_subgrid, approx_facet, ...)	Write approx subgrid and facet to hdf5

ska_sdp_exec_swiftly.fourier_transform_dask Module

Main algorithm routine. The functions that conduct the main Dask-implemented algorithm include the subgrid to facet, and facet to subgrid transformations. The main function calls all the functions.

Functions

`add_two`(one, two, **kwargs)	Functions for iterative operations to accelerate the construction of graphs
`cli_parser`()	Parse command line arguments
`errors_facet_to_subgrid_2d`(NMBF_NMBF, ...[, ...])	Calculate the error terms for the 2D facet to subgrid algorithm.
`errors_facet_to_subgrid_2d_dask`(...)	Functions for calculating the error of approx subgrid
`errors_subgrid_to_facet_2d`(BMNAF_BMNAF, ...)	Calculate the error terms for the 2D subgrid to facet algorithm.
`errors_subgrid_to_facet_2d_dask`(...)	Functions for calculating the error of approx facets
`facet_to_subgrid_2d_method_1`(facet, ...[, ...])	Generate subgrid from facet 2D.
`facet_to_subgrid_2d_method_2`(facet, ...[, ...])	Approach 2: First, do prepare_facet on the horizontal axis
`facet_to_subgrid_2d_method_3`(facet, ...[, ...])	Generate subgrid from facet 2D.
`generate_approx_subgrid`(NMBF_NMBF, ...[, ...])	Finish generating subgrids from facets.
`generate_input_data`(sparse_ft_class[, ...])	Generate standard data G and FG
`main`(args)	Main function to run the Distributed FFT
`make_subgrid_and_facet`(G, FG, base_arrays, dims)	Calculate the actual subgrids and facets.
`make_subgrid_and_facet_from_hdf5`(G, FG, ...)	Calculate the actual subgrids and facets.
`make_subgrid_and_facet_from_sources`(sources, ...)	Calculate the actual subgrids and facets from a list of specific sources.
`plot_pswf`(constants_class[, fig_name])	Plot to check that PSWF indeed satisfies intended bounds.
`plot_work_terms`(constants_class[, fig_name])	Calculate actual work terms to use and plot to check them.
`run_distributed_fft`(fundamental_params[, ...])	Main execution function that reads in the configuration, generates the source data, and runs the algorithm.
`set_up_dask`([scheduler_address])	Set up the Dask Client
`subgrid_to_facet_algorithm`(subgrid_2, ...[, ...])	Generate facets from subgrids.
`tear_down_dask`(client)	Close the Dask Client
`write_hdf5`(approx_subgrid, approx_facet, ...)	Write approx subgrid and facet to hdf5

ska_sdp_exec_swiftly.generate_hdf5 Module

Small script for generating hdf5 test files, including FG, G

Functions

`cli_parser`()	Parse command line arguments
`dask_wrapper`(func)	The Dask wrapper function
`direct_ft_chunk_work`(G_2_path, chunk_slice, ...)	Calculate the value of a chunk of direct fourier transform and
`generate_data_hdf5`(npixel, G_2_path, ...[, ...])	Generate standard data G and FG with hdf5
`main`(args)	Main function to generate G and FG hdf5 file
`make_subgrid_from_sources`(sources, ...[, ...])	Generates a subgrid from a source list
`set_up_dask`([scheduler_address])	Set up the Dask Client
`tear_down_dask`(client)	Close the Dask Client