performance_analysis

performance_analysis is a command line app written using RASCIL. It helps in analysis of performance files written by rascil_imager.

The performance files can be obtained using a script to iterate over some parameter. For example:

#!/usr/bin/env bash
#
results_dir=${HOME}/results/5km_resource_modelling
for int_time in 2880 1440 720 360
  do
    mshome=${HOME}/data/int_time${int_time}
    for npixel in 512 1024 2048 4096 8192
    do
      results_dir=${HOME}/data/int_time${int_time}_npixel${npixel}
      mkdir -p ${results_dir}
      python3 ${RASCIL}/rascil/apps/rascil_imager.py --mode cip \
        --clean_nmoment 3 --clean_facets 4 --clean_nmajor 10 \
        --clean_threshold 3e-5 --clean_restore_facets 4 --clean_restore_overlap 32 \
        --use_dask True --imaging_context ng --imaging_npixel ${npixel} --imaging_pol stokesI --clean_restored_output list \
        --imaging_cellsize 5e-6 --imaging_weighting uniform --imaging_nchan 1 \
        --ingest_vis_nchan 100 --ingest_chan_per_vis 16 \
        --ingest_msname ${mshome}/SKA_MID_SIM.ms \
        --performance_file ${results_dir}/performance_rascil_imager_${int_time}_${npixel}.json
    done
  done

In addition, the memory usage can be tracked using a dask plugin. Currently this requires setting up the dask scheduler with the plugin:

ssh $scheduler dask-scheduler --port=8786 --preload dask_memusage --memusage-csv \
./performance_rascil_imager_${1}_${2}.csv  &

Command line arguments

RASCIL performance analysis

usage: performance_analysis [-h] [--mode MODE]
                            [--performance_files [PERFORMANCE_FILES ...]]
                            [--memory_file MEMORY_FILE] [--tag TAG]
                            [--parameters [PARAMETERS ...]]
                            [--functions [FUNCTIONS ...]]
                            [--vis_nvis VIS_NVIS] [--verbose VERBOSE]
                            [--results RESULTS]

Named Arguments

--mode

Processing mode: line | bar | contour | summary | fit

Default: “summary”

--performance_files

Names of json performance files to analyse: default is all json files in working directory

--memory_file

Name of memusage csv file

--tag

Informational tag used in plot titles and file names

Default: “”

--parameters

Name of parameters from cli_args e.g. imaging_npixel_sq, used for line (1 parameter) and contour plots (2 parameters)

Default: [‘imaging_npixel_sq’, ‘vis_nvis’]

--functions

Names of values from dask_profile to plot e.g. skymodel_predict_calibrate

Default: [‘skymodel_predict_calibrate’, ‘skymodel_calibrate_invert’, ‘invert_ng’, ‘restore_cube’, ‘image_scatter_facets’, ‘image_gather_facets’]

--vis_nvis

Number of visibilities for use if vis_nvis not in json files

--verbose

Verbose output?

Default: “False”

--results

Directory for results, default is current directory

Default: “./”