SKA SRC Software Discovery API

Science-ready software catalog that registers and discovers metadata for SKA software containers. Delivered as a FastAPI microservice with PostgreSQL persistence, Alembic migrations, and Helm packaging.

Features

  • Youcat integration — Integration with Youcat for fetching software metadata.

  • Software Retrieval by URI — Look up a registration by URI via GET /v1/registrations?software_uri=<uri>.

  • Software Retrieval by ID — Look up a registration by its UUID via GET /v1/registrations/{registration_id}.

  • Software Ingestion — Ingest a new software or update an existing software metadata via POST /v1/ingest.

  • Software Patch — Patch certain parts of the existing software metadata (Strictly follows RFC-7396) PATCH /v1/registrations/{registration_id}.

  • Software Deletion — Remove existing entries via DELETE /v1/registrations/{registration_id}.

  • Metadata Validation — Validate a payload against the SDM schema (read-only) via POST /v1/validate.

  • Health Monitoring — /v1/health reports uptime, request count, and dependent service status.

Architecture

Architecture diagram

The Software Discovery API sits alongside YouCat (a TAP service) in front of a shared PostgreSQL database that holds the sdm.* metadata catalogue:

  • SDAPI core — handles write/administrative operations (POST /v1/ingest, PUT /v1/software/{uri}, DELETE /v1/registrations/{registration_id}) and enforces authentication/RBAC via IAM token validation.

  • YouCat (TAP service) — exposes the sdm.* schema for read-heavy discovery via ADQL/TAP queries, with a UWS job manager for asynchronous query execution.

  • PostgreSQL — the shared backing store queried/written by both SDAPI core and YouCat.

Example flow — ingesting and then discovering software:

  1. A CLI/TAP client sends POST /v1/ingest with a Software payload to SDAPI core.

  2. SDAPI core validates the IAM token, persists the metadata into the sdm.* schema in PostgreSQL, and returns the generated registration id.

  3. A separate client later queries YouCat’s TAP endpoint with an ADQL query (e.g. SELECT * FROM sdm.software WHERE uri = 'ska:software:dsc-037@0.1.3') to discover the same record without going through SDAPI core.

API Endpoints

Health Check

GET /v1/health

Returns uptime, request count, and status of dependent services.

Software Data Model

Please refer to the Software Data Model repo for a detailed overview of the Software Data Model.

Get Software Registration

GET /v1/registrations?software_uri=<uri>
Authorization: Bearer <token>

Retrieve the full Software metadata record that matches the supplied software_uri query parameter.

Query Parameter:

Parameter

Required

Description

software_uri

✅ Yes

Fully-qualified URI of the software to retrieve. Format: {publisher}:{name}@{semver}, e.g. ska:software:dsc-037@0.1.3. Max 255 chars.

Example request:

GET /v1/registrations?software_uri=ska%3Asoftware%3Adsc-037%400.1.3
Authorization: Bearer <token>

Response — success (200 OK):

{
  "uri": "ska:software:dsc-037@0.1.3",
  "description": "EoR cable reflection systematics analysis pipeline for SKA Low data",
  "release_date": "2025-11-25T00:00:00Z",
  "status": "STABLE",
  "artifacts": [
    {
      "kind": "DOCKER",
      "location": "https://images.canfar.net/dsc-037-delay-ps:0.1.3",
      "cpu_architecture": [
        "amd64",
        "arm64"
      ],
      "digest": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
      "supported_modes": [
        "NOTEBOOK",
        "DESKTOP",
        "HEADLESS"
      ]
    }
  ]
}

Status

When

200

Registration found and returned

401

Missing or invalid authentication token

404

No registration exists for the given software_uri

500

Unexpected internal error

Get Software Registration by ID

GET /v1/registrations/{registration_id}
Authorization: Bearer <token>

Retrieve the full Software metadata record that matches the supplied registration_id path parameter. The registration_id is the UUID assigned to the registration at ingest time.

Path Parameter:

Parameter

Required

Description

registration_id

✅ Yes

UUID of the software registration to retrieve, e.g. 3fa85f64-5717-4562-b3fc-2c963f66afa6.

Example request:

GET /v1/registrations/3fa85f64-5717-4562-b3fc-2c963f66afa6
Authorization: Bearer <token>

Response — success (200 OK):

{
  "uri": "ska:software:dsc-037@0.1.3",
  "description": "EoR cable reflection systematics analysis pipeline for SKA Low data",
  "release_date": "2025-11-25T00:00:00Z",
  "status": "STABLE",
  "artifacts": [
    {
      "kind": "DOCKER",
      "location": "https://images.canfar.net/dsc-037-delay-ps:0.1.3",
      "cpu_architecture": [
        "amd64",
        "arm64"
      ],
      "digest": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
      "supported_modes": [
        "NOTEBOOK",
        "DESKTOP",
        "HEADLESS"
      ]
    }
  ]
}

Status

When

200

Registration found and returned

401

Missing or invalid authentication token

404

No registration exists for the given registration_id

500

Unexpected internal error

Ingest Software

POST /v1/ingest
Authorization: Bearer <token>
Content-Type: application/json

Request Contract — Software Ingestion API :

### 1. Required Fields (Hard Validation)

The following fields must be present. Requests missing these will be rejected with `400 Bad Request`.

| Field                          | Type         | Rules                                                                      |
| ------------------------------ | ------------ | -------------------------------------------------------------------------- |
| `uri`                          | URI          | Format: {publisher}:{name}:{semver}, e.g., ska:software:dsc-037-delay-ps:0.1.3     |
|                                |              | Max 255 chars. Must be unique per version.                                 |
| `artifacts`                    | array        | At least one artifact required                                             |
| `artifacts[].kind`             | string       | Must be one of the allowed values listed below                             |
| `artifacts[].location`         | string       | Non-empty string                                                           |
| `artifacts[].cpu_architecture` | array        | At least one value required; allowed: "amd64", "arm64"                     |

---

### 2. Optional Fields (Soft Metadata)

These fields improve discovery but are not mandatory.

* `description` — string
* `release_date` — IVOA DALI UTC timestamp (e.g. `2025-11-25T00:00:00Z`)
* `changelog` — string
* `status` — one of the allowed status values listed below
* `discovery` — object (defaults to empty lists)
* `data_compatibility` — object (defaults to empty lists)
* `resources` — object (defaults to `requires_gpu: false`, null memories)
* `provenance.repository_url` — string
* `provenance.registered_by` — string
* `provenance.registration_date` — IVOA DALI UTC timestamp
* `artifacts[].digest` — string matching `sha256:<64 hex chars>`
* `artifacts[].entrypoint` — string
* `artifacts[].supported_modes` — array of allowed execution modes

If omitted, database defaults or `NULL` values are applied.

---

### 3. Allowed Values (Controlled Fields)

#### Software Status

Allowed values:

* `ALPHA`
* `BETA`
* `TESTING`
* `STABLE`
* `DEPRECATED`

Values are case-sensitive and must be sent exactly as above.

#### Artifact Type

Allowed values:

* `DOCKER`
* `SINGULARITY`
* `OCI`

#### Supported Execution Modes (inside artifacts)

Allowed values:

* `NOTEBOOK`
* `DESKTOP`
* `HEADLESS`

Multiple values may be supplied as an array.

---

### 4. Structural Constraints

| Field              | Constraint                                              |
| ------------------ | ------------------------------------------------------- |
| `uri`              | Unique identifier, 5–255 chars                          |
| `artifacts`        | At least one required (one-to-many relationship)        |
| `cpu_architecture` | At least one value required per artifact                |
| Memory values      | Integers ≥ 1 (interpreted as GB)                        |
| `requires_gpu`     | Defaults to `false`                                     |
| Timestamps         | IVOA DALI format, UTC only (Z or +00:00)                |
| 1:1 sections       | Only one record allowed per software                    |
| Extra fields       | Not permitted (strict schema)                           |

---

Request Body:

{
  "uri": "ska:software:sapid-ocs@1.0.0",
  "description": "Service responsible for orchestration, scheduling, and execution of observation control workflows within SAPID.",
  "release_date": "2026-01-10T00:00:00Z",
  "changelog": "Initial OCS integration release with scheduling and telemetry hooks.",
  "status": "STABLE",
  "discovery": {
    "science_category": [
      "Observatory Operations"
    ],
    "function_category": [
      "Observation Scheduling",
      "Workflow Orchestration",
      "Telemetry Integration"
    ],
    "science_working_group": [
      "Operations"
    ],
    "tools_included": [
      "FastAPI",
      "PostgreSQL",
      "SQLAlchemy",
      "Prometheus",
      "Pydantic"
    ]
  },
  "data_compatibility": {
    "data_input_type": [
      "Observation Requests",
      "Scheduling Blocks",
      "Instrument Configuration"
    ],
    "data_output_type": [
      "Execution Logs",
      "Observation Status",
      "Telemetry Events"
    ]
  },
  "resources": {
    "requires_gpu": false,
    "min_memory": 8,
    "recommended_memory": 16
  },
  "artifacts": [
    {
      "kind": "DOCKER",
      "location": "registry.example.org/sapid-ocs:1.0.0",
      "cpu_architecture": [
        "amd64"
      ],
      "digest": "sha256:placeholderdigest123456789",
      "entrypoint": "uvicorn app.main:app --host 0.0.0.0 --port 8080",
      "supported_modes": [
        "HEADLESS",
        "NOTEBOOK"
      ]
    }
  ],
  "provenance": {
    "repository_url": "https://gitlab.example.org/sapid/sapid-ocs",
    "registered_by": "sapid-system",
    "registration_date": "2026-01-10T00:00:00Z"
  }
}

Response:

{
  "id": "870d81f7-4d31-44a8-855c-a2118c2731a8"
}

PATCH Software

Endpoint

PATCH /v1/registrations/{registration_id}

Headers

Authorization: Bearer <token>
Content-Type: application/json

Request Body

A partial software metadata document following RFC 7396 (JSON Merge Patch). Only the fields to be updated need to be included.

Example:

{
  "description": "Updated description",
  "status": "DEPRECATED"
}

To clear an optional field:

{
  "description": null
}

Arrays are replaced in their entirety. For example:

{
  "artifacts": [
    {
      "kind": "DOCKER",
      "location": "registry.example.org/example:1.0.1",
      "cpu_architecture": ["amd64"]
    }
  ]
}

Response (200 OK)

Returns the complete updated metadata record.

{
  "uri": "ska:software:sapid-ocs@1.0.0",
  "description": "Updated description",
  "status": "DEPRECATED",
  "artifacts": [
    {
      "kind": "DOCKER",
      "location": "registry.example.org/example:1.0.1",
      "cpu_architecture": ["amd64"]
    }
  ],
  "created_at": "2026-02-16T12:00:00.000000"
}

Error Responses

  • 404 Not Found – Registration not found.

  • 422 Unprocessable Entity – The merged document is invalid.


### Delete Software

```http
DELETE /v1/registrations/{registration_id}
Authorization: Bearer <token>

Path Parameter:

  • registration_id: Unique UUID of the software registration to delete (for example: 123e4567-e89b-12d3-a456-426614174000)

Response:

{
  "registration_id": "123e4567-e89b-12d3-a456-426614174000",
  "message": "Software '123e4567-e89b-12d3-a456-426614174000' deleted successfully."
}

Validate Software Metadata

POST /v1/validate
Authorization: Bearer <token>
Content-Type: application/json

Validates a raw Software JSON document against the SDM Pydantic schema. Read-only — no data is persisted to the database.

Request Body: (same structure as POST /v1/software — any JSON object is accepted)

{
  "uri": "ska:eor-delay-ps:0.1.3",
  "description": "EoR cable reflection systematics analysis pipeline for SKA Low data",
  "release_date": "2025-11-25T00:00:00Z",
  "status": "STABLE",
  "artifacts": [
    {
      "kind": "DOCKER",
      "location": "https://images.canfar.net/dsc-037-delay-ps:0.1.3",
      "cpu_architecture": [
        "amd64",
        "arm64"
      ],
      "digest": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
      "supported_modes": [
        "NOTEBOOK",
        "DESKTOP",
        "HEADLESS"
      ]
    }
  ]
}

Response — valid payload (200 OK):

{
  "valid": true,
  "errors": []
}

Response — invalid payload (422 Unprocessable Entity):

{
  "error": {
    "code": "VALIDATION_ERROR",
    "errors": [
      "uri: field required",
      "artifacts.0.kind: value is not a valid enumeration member"
    ]
  }
}

Status

When

200

Payload satisfies the SDM schema

422

One or more SDM schema violations found

500

Unexpected internal error

Repository Structure

  • src/ - Main application code including API routes, models, and services.

  • alembic/ - Database migration scripts.

  • charts/ - Helm charts for deploying the service.

  • tests/ - Unit and integration tests.

  • etc/ - Configuration files.

  • images/ - Dockerfile and related container files.

Getting Started

Prerequisites

  • Python 3.13+

  • PostgreSQL 12+

  • Docker (for containerization)

  • Helm (for deployment)

Installation

  1. Clone the repository:

    git clone https://gitlab.com/ska-telescope/src/src-mm/ska-src-mm-software-discovery.git
cd ska-src-mm-software-discovery/etc/helm

  2. Update configuration files in etc/helm/values.yaml as needed.

  3. Deploy using Helm:

    helm install ska-src-software-discovery ./ska-src-software-discovery

Running Locally

  1. Update .env file with database and other configurations.

  2. Run docker-compose:

    docker-compose up --build

  3. Exec into the container and run migrations:

    docker exec -it sdapi-core bash
alembic upgrade heads

  4. Access the youcat API at http://localhost:8080/tap.

  5. Access the software discovery API at http://localhost:8081/v1.