AWS DP HPC cluster
==================

This page provides an overview of the storage areas available on the AWS DP HPC cluster and their
intended usage.

Storage areas
-------------

The AWS DP HPC cluster provides three types of storage:

- **Home Directory (``~/``)**

  - Path: ``/home/<username>``
  - Space: total 1968 GB
  - Purpose: scripts, configuration files, small personal files
  - Note: limited space; do not store large datasets

- **Shared Storage (``/shared/fsx1/``)**

  - Space: total 28970 GB
  - Purpose: high-performance storage accessible by all compute nodes
  - Use for: pipeline inputs, temporary working data, outputs
  - Note: short-term storage; may be cleared during cluster maintenance

- **S3 (``s3://<skao-sdp-testdata>/``)**

  - Purpose: long-term cloud storage
  - Use for: test datasets and archiving results
  - Note: persists beyond cluster lifetime