Imaging I/O test¶

This module contains the functions to run Imaging IO benchmark.

sdpbenchmarks.imagingiobench.check_iotest_arguments(conf)[source]¶

Checks the arguments passed in the config file

Parameters: conf (dict) – A dict containing configuration.
Returns: Error code 0 OK, 1 Not OK
Return type: int

sdpbenchmarks.imagingiobench.compile_imaging_iotest(conf)[source]¶

Compiles Imaging IO test by cloning the code from Git

Parameters: conf (dict) – A dict containing configuration.
Returns: 0 OK, 1 Not OK
Return type: int
Raises: ImagingIOTestError – An error occurred during compiling of the code

sdpbenchmarks.imagingiobench.create_bench_conf(tag, run_mode, num_nodes, rep, rec_set, vis_set, chunk_sizes)[source]¶

Creates a dict containing the parameters for a given run

Parameters

tag (str) – Tag of the benchmark run (given at the CLI)
run_mode (str) – Mode of the run
num_nodes (int) – Number of nodes
rep (int) – Number of repetitions
rec_set (str) – Image size
vis_set (str) – Antennas configuration
chunk_sizes (str) – Chunk sizes for time and frequency domains

Returns

A dict containing all the parameters

Return type

dict

sdpbenchmarks.imagingiobench.extract_metrics(filename, mpi_startup)[source]¶

Extract data transfer metrics from benchmark output

Parameters

filename (str) – Name of the stdout file to parse
mpi_startup (str) – Startup command of MPI (mpirun or srun)

Returns

A dict containing streamer, producer and writer metrics

Return type

dict

sdpbenchmarks.imagingiobench.get_command_to_execute_bench(conf, param)[source]¶

This function forms the command string to be executed

Parameters

conf (dict) – A dictionary containing configuration parameters
param (dict) – A Parameter dict with all relevant arguments for the run

Returns

Complete command string

Return type

str

sdpbenchmarks.imagingiobench.get_mpi_args(conf, num_nodes, num_omp_threads, num_processes)[source]¶

Extract all MPI specific arguments and form a string

Parameters

conf (dict) – A dict containing the config parameters of benchmark
num_nodes (int) – Number of nodes
num_omp_threads (int) – Number of OpenMP threads
num_processes (int) – Number of MPI processes

Returns

A string containing all MPI related arguments

Return type

str

sdpbenchmarks.imagingiobench.get_num_processes(conf, rec_set, num_nodes)[source]¶

Estimates producers, streamers OpenMP threads.

Parameters

conf (dict) – A dict containing configuration settings
rec_set (str) – image parameters
num_nodes (int) – number of nodes

Returns

total cpu cores (only physical) on all nodes combined, threads per each core, number of OpenMP threads, number of producers, number of MPI processes

Return type

list

sdpbenchmarks.imagingiobench.get_telescope_config_settings(param)[source]¶

This function returns the telescope related configurations

Parameters: param (dict) – Parameter class with all relevant arguments for the run
Returns: String with time, frequency and w-stacking arguments
Return type: str

sdpbenchmarks.imagingiobench.prepare_iotest(conf)[source]¶

Prepare IO Imaging Benchmark installation.

Parameters: conf (dict) – A dict containing configuration.
Returns: 0 OK , 1 Not OK
Return type: int

sdpbenchmarks.imagingiobench.print_key_stats(run_prefix, metrics)[source]¶

This prints the key metrics to stdout

Parameters

run_prefix (str) – Prefix of the run
metrics (dict) – A dict containing stats of the Imaging IO bench

sdpbenchmarks.imagingiobench.run_iotest(conf)[source]¶

Run Imaging IO test.

Parameters: conf (dict) – A dict containing configuration.
Returns: < 0 all runs failed, = 0 all runs passed, > 0 few runs passed
Return type: int
Raises: Exception – An error occurred in building parameter sweep, running or submitting job