CHANGELOG

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Development

Added

• Development Container

[0.8.0] 2025-06-06

Added

• Introduced the MonteCarloSimulations class for handling setup and execution of Monte Carlo simulations.

Changed

• Reimplemented the sampling algorithm for Monte Carlo Simulations to utilise frozen distributions from scipy.stats in order to ensure consistent sampling. Added Komolgorov-Smirnov tests to check that sampled values are from the expected distributions as per configuration.

• The following parameter names in the configuration model for simulated pipelines have been renamed to avoid confusion between the sampling distribution hyperparameters and the sampled value during Monte Carlo simulations:

  • node_hours —> node_hours_mean

  • pct_parallelism —> pct_parallelism_min

• Improved code readability and documentation.

[0.7.0] 2025-05-08

Added

• End-to-end simulation notebook which uses a set of scientific parameters for different projects as input and calls the SDP Parametric Model to generate inputs for the SDP Resource Model, which then runs a hardware optimisation over a 12 week observing schedule and a Monte Carlo simulation over a 1 week schedule. Additional plots and statistics are also included to estimate the load on the SDP.

• Option to use Dask to distribute the optimisation trials. Note that this can cause issues on some machines if SQLite is used as the database backend (which is the default) – this will be addressed in a future release.

• Notebook to generate an observing schedule from a set of scheduling block types and pipeline configurations.

• Plots to visualise the number of nodes per pipeline group suggested by the optimisation step in the notebook showing how to run the optimisation with monte carlo simulations.

Changed

• Capacity storage for raw visibilities is now allocated before observing starts for each scheduling block instance individually. These are retained until all processing for the last scheduling block instance completes with no data retention period.

• Data product storage is now a separate resource rather than being part of capacity storage.

• Storage for data products is allocated before each pipeline starts and released individually following a data retention period set by the scheduling block type config.

• Only one set of pipelines (with performance storage allocated and released) is run for a scheduling block once observations for each scheduling block instance are completed. Node hours and storage requirements for each pipeline are assumed to represent the entire scheduling block.

• Rather than suggesting values and then normalising to the budget (which makes it harder for the optimisation algorithm to find good values) the optimisation step will now split the budget hierarchically as follows:

• Fixed a bug in the function that generates IDs for scheduling block instances that caused instances to be incorrectly grouped together into a scheduling block when there were more than two instances of the same type in an observing schedule.

• Added the option to run the Monte Carlo simulations in parallel to increase performance. A new CLI parameter num_workers has been added to support this.

  1. Suggest an amount to spend on compute (minimum = cost per node, maximum = 90% of the total budget // cost per node, in increments of the cost per node (avoids wasting part of the budget and avoids sampling these values in the first place)

  2. Suggest an amount from the remaining budget to spend on performance storage (between 10-90% of the remaining budget)

  3. Suggest an amount from the remaining budget to spend on capacity storage (between 10-90% of the remaining budget)

  4. Use the rest of the budget on data product storage

• Fixed a bug in the function that generates IDs for scheduling block instances that caused instances to be incorrectly grouped together into a scheduling block when there were more than two instances of the same type in an observing schedule.

• Increased cost per compute node

• Replaced default config files with low AA2 SV data (derived from end-to-end notebook that uses the parametric model)

Removed

• Removed notebooks in the experiments directory that have been replaced by the end-to-end simulation notebook.

[0.6.0] 2025-04-23

Added

• Functionality to simulate running pipelines in parallel and/or in series according to new priority parameter in pipeline configuration file.

Changed

• Capacity storage for all scheduling block instances is now allocated at the beginning of a scheduling block before the first observation is executed rather than at the beginning of processing. This simulates the situation where observations will be delayed until the SDP can store the data. The Gantt charts reflect this change. Note that this will be modified in a future release so that storage is allocated before each scheduling block instance rather than scheduling block.

• Performance improvements to the process_inputs, resource_usage, monte_carlo and optimise modules to speed up the simulation.

• Pathlib Path objects are used in notebooks.

[0.5.0] 2025-04-10

Added

• CLI for running an experiment that includes both hardware optimisation and a Monte Carlo simulation on a given set of inputs.

• Example SLURM script for running an experiment on AWS.

• Configuration files and Jupyter notebooks for running example experiments.

• Tutorial on RTD for running an experiment.

Changed

• The observation mix csv input file with multiple data columns has been replaced by a json file. This simply lists the scheduling block types to be run in the order they are to be run. The simulation will assemble the configuration for each scheduling block instance using the information in the scheduling block types and pipeline configuration files.

• Observation mix has been renamed to observing schedule for clarity.

• Pathlib Path objects are used for all file paths.

[0.4.0] 2025-03-31

Added

• Monte Carlo simulation module (ska_sdp_resource_model.simulate.monte_carlo) for sampling the percentage parallelism and node hours of each pipeline and shuffling the order of observations.

• Option to generate an observation schedule when running a simulation with or without Monte Carlo simulations given a total time in hours to generate. This is done once at the start of the run by randomly sampling scheduling block types from the scheduling block types configuration file until the total time is reached.

Changed

• Pipeline configurations (in pipelines.json) now require uncertainty of node hours (node_hours_uncertainty) and maximum percentage parallelism (pct_parallelism_max) for running Monte Carlo simulations.

• Scheduling block types configurations (in scheduling_block_types.json) now require a short name (short_name) for generating IDs if generating observation mixes.

• Storage for raw visibilities and data products is now allocated for all instances in a scheduling block when the first instance begins processing and is released when the final instance has completed. If the storage requirements for any block exceed the capacity given by the hardware configuration, the simulation will fail.

[0.3.0] 2025-03-17

Added

• Optimisation module (ska_sdp_resource_model.optimise.optimise) for optimising the hardware configuration and nodes allocated to each pipeline for a given observation mix

[0.2.0] 2025-02-18

Added

• Text summary of total time for observation mix to complete to Dash app

• Strip plot of waiting times for each event in the simulation to Dash app

• Gantt-style plot of total time for each scheduling block instance to complete to Dash app

• Treemap showing total node hours and percentage parallelism for each scheduling block type and pipeline to Dash app

• Python module for plotting functions

• Functions to process outputs from simulation

Changed

• Move processing and plotting code blocks from notebook to python modules

[0.1.1] 2025-02-13

Changed

• Use official product names for pipeline names

• Update Jupyter notebook with version used in PI25.5 system demo

• Rename data_product_storage in pipeline config to data_product_storage_gb to be consistent with other storage naming that includes units

• Remove unnecessary prefixes on storage wait event names

Fixed

• Fix “file name too long” error when uploading custom observation mix file in dash app by saving contents to temporary file

• Fix logging so that –verbose flag on CLI results in logging to console

[0.1.0] 2025-02-11

• Initial release of the SDP Resource Model

Added

• Initial release

• Simulation library

• Dash app

• CLI

• Default configuration files

• Jupyter notebook