How It Works#

Understand the SKAO Vault structure, KV engines, and access control.

Vault as a secure configuration store#

SKAO treats Vault as a secure store for configurations. This approach provides:

Consistent management of both sensitive and non-sensitive configuration
Unified access control across all configuration data
Audit trails for tracking access and changes
Single source of truth for deployments across datacentres

Note

While we use “configurations” and “secrets” interchangeably, secrets still require stricter handling and must never be leaked.

Understanding the structure#

SKAO organises Vault data across three levels of ownership:

Level	Description
Datacentre	Physical infrastructure hosting multiple environments
Environment	Set of applications/services deployed together (e.g., staging, production)
Application/Service	Individual software component or infrastructure piece

This hierarchy enables tight access control and efficient management at scale.

Path structure#

Every Vault secret follows this pattern:

<prefix>/<service>/<secret>

Where:

<prefix> — Context path (e.g., shared/global, <datacentre>/<environment>, dev/<team>)
<service> — Application, service, or infrastructure component name
<secret> — Specific configuration or secret (credentials, config files, Helm values)

Example in SKAO context:

KV engines#

Vault organises secrets into KV (key-value) engines. SKAO uses three categories:

Engine	Purpose
Dev	Development deployments and CI/CD
Shared	Cross-datacentre configurations (production-grade)
Datacentre-specific	Per-datacentre configurations with environment segregation

Separating engines enables:

Simplified organisation and management
Easier access control policies
Reduced duplication through shared sections

Dev KV engine#

The Dev engine serves as a “meta-engine” for development teams, organising by team rather than datacentre.

Per-team section:

Path: dev/<team>/*/<service>/
Contents: Team-specific secrets for development, integration, and staging
Users: SRE, AIV, and Development teams

Dev-shared section:

Path: dev/shared/
Contents: Secrets shared across teams for development environments
Users: SRE, AIV, and Development teams

Example structure:

dev/
└── some-team/
    ├── some-service/
    │   └── values.yml
    ├── another-service/
    │   └── database_credentials
    └── infra-component/
        └── config.yml

Note

Your team’s slug must match your GitLab group at ska-telescope/ska-dev. Raise a ticket on the System Team Support Desk if you need a group created.

Shared KV engine#

The Shared engine stores configurations used across multiple datacentres or environments.

Global section:

Path: shared/global/
Contents: Secrets common to all datacentres and environments
Examples: External API keys, infrastructure configurations, common firewall rules
Users: SRE and AIV teams

Environment-type sections:

Path: shared/<environment>/ or shared/<environment-type>/
Contents: Secrets shared among environments of the same type
Users: SRE and AIV teams

Example structure:

shared/
  ├── global/                 # All datacentres
  ├── production/             # All production environments
  └── staging/                # All staging environments

Note

The environment-type construct (e.g., shared/dish-lmc) enables sharing among differently-named but similar environments like dish-lmc-001, dish-lmc-002, etc.

Warning

Changes to shared sections affect all environments using them. We enforce strict, fine-grained access control.

Datacentre-specific KV engines#

Each datacentre has its own KV engine with environment-based organisation.

Per-environment section:

Path: <datacentre>/<environment>/
Contents: Environment-specific configurations
Examples: Database configs, network settings, infrastructure component configs
Users: SRE and AIV teams

Datacentre-shared section:

Path: <datacentre>/shared/
Contents: Secrets shared across environments within the datacentre
Users: SRE and AIV teams

Best practices#

Use the appropriate engine:

Development and CI/CD → dev/<team>/
Shared across environments → shared/
Production deployments → <datacentre>/<environment>/

Follow the path pattern:

Always use <prefix>/<service>/<secret>
Create subdirectories — root paths don’t allow direct secrets

Minimise shared section changes:

Changes propagate to all consuming environments
Test thoroughly before modifying shared configurations

Replace GitLab CI variables:

Store secrets in Vault instead of GitLab CI variables
Use Vault how-to guides for GitLab integration patterns

Deprecation notices#

Deprecated paths:

kv/users/groups/ska-dev/<team>/ → Use dev/<team>/ instead
Removal in Sprint #2 of PI 25

Deprecated integrations:

Vault Sidecar Injector → Use Vault Secrets Operator
Vault CSI Driver → Use Vault Secrets Operator
Support ends Sprint #2 of PI 25
ska-tango-util v0.4.13+ migrates automatically