Deploying packages to AWS

Conventions

See also the general conventions for this repository.
ska-sdp-spack versions have the form <year>.<month>.<increment>, where the month always has two digits. For example, 2026.02.5 is the fifth release in February 2026.

Deploying a new release

Deploying packages to the DP AWS HPC Cluster works by creating a new ska-sdp-spack release. See also the standard SKAO release process.

Create a new Release Management JIRA ticket.
Create a release branch, e.g., rel-42-2025.02.1 if the JIRA ticket is REL-42 and the new version is 2025.02.1.
- Update the CHANGELOG.md. Ensure that the changes since the last release are there.
- Update the version according to the version conventions.
  - The standard make bump-minor-release and make bump-patch-release commands may not work well if the month starts with a zero.
  - A basic search-and-replace can easily update all occurrences of the version in the repository.
  - When in doubt, peek at a merge request for an older release.
Create a merge request for the release branch. See the Git workflow.
After merging the merge request, create a tag for the new release. If you’re not a maintainer of ska-sdp-spack, ask one of the maintainers, e.g., in the #ska-sdp-spack Slack channel.
The CI build for the tag will include a deploy-aws job which automatically deploys all packages in the env/aws environment to AWS.

Using released packages

The module command allows using packages from a deployed release on AWS:

# Make ska-sdp-spack modules available.
module use /shared/fsx1/shared/metamodules
# Make the modules in the 2025.02.1 release available.
module --no-pager load ska-sdp-spack/2025.02.1
# Load a deployed package, including all dependencies.
module --no-pager load py-rapthor

The --no-pager argument is especially useful when using these commands in a script, e.g., a SLURM sbatch script that runs a pipeline.

The pipelines cookbook has more details about loading Spack modules.