# ska-src-ms-system-testing-suite

Centralized System testing suite for SKA services. This repository provides behavior-driven tests, reusable test utilities, and tooling to run, validate, and document system-level flows.

**Note:** For full documentation, see the [System Testing Suite Docs](https://developer.skao.int/projects/ska-src-ms-system-testing-suite/en/latest/).

## Features

- Python 3.10 with Poetry for dependency management and reproducible environments
- pytest-based test runner with BDD-style feature files
- Utilities for IAM-backed authentication and service API calls
- Makefile-driven local workflows (optional)
- Helm chart for containerized execution
- Docker Compose for local development and testing

## Requirements

- Python 3.10
- Poetry
- Helm, Docker and Docker Compose (optional, for containerized runs)

## Quick start

- Install dependencies:
  - `poetry install`
- Configure environment:
  - `cp .env.template .env`
  - Edit `.env` as needed (see [Configuration](#configuration-1) section)
- Run tests:
  - `docker-compose up --build`

## Deployment

### Prerequisites
- Kubernetes cluster
- Helm installed
- Access to container registry

### Installation

```shell
  helm install system-testing ./etc/helm -f your-values.yaml
```

### Configuration

**Authentication**

The application uses IAM client credentials for authentication. These are configured in the Helm values file:

```yaml
deployment:
  system_testing_suite:
    image:
      registry: example.registry.com
      image: ska-src-ms-system-testing-suite
      tag: latest
      pullPolicy: IfNotPresent

    iam_client:
      client_secret:
        name: iam-client-credentials
        id: <iam_client_id>
        secret: <iam_client_secret>
        existingSecret: false
      user_credentials:
        name: iam-user-credentials
        user_id: <user_id>
        password: <user_password>
        existingSecret: false
```

To use existing Kubernetes secrets instead, set existingSecret to true and ensure secrets with the appropriate names exist in the namespace.

**RSE Configuration**

The application requires RSE - File mappings for test execution. These are configured in the Helm chart's ConfigMap template (`charts/ska-src-ms-system-testing-suite/templates/config.yaml`):
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: test-configmap
data:
  config.json: |
    {
        "CHSRC_XRD_PROD": "file in the CHSRC_XRD_PROD rse",
        "CHSRC_XRD_DEV": "file in the CHSRC_XRD_DEV rse",
    }
```
To update the RSE mappings, modify the `config.json` data in the ConfigMap template and apply the changes with:
```shell
  kubectl apply -f charts/ska-src-ms-system-testing-suite/templates/config.yaml
```
or 

```shell
  helm upgrade system-testing ./charts/ska-src-ms-system-testing-suite --reuse-values
```

## Configuration

The suite reads configuration from environment variables and a `.env` file in the repository root. Common variables to set:
- `IAM_CLIENT_CONF_URL`: Well-known IAM discovery URL.
- `API_IAM_CLIENT_ID`: Client ID for authentication.
- `API_IAM_CLIENT_SECRET`: Client secret for authentication.
- `USER_TOKEN`: User access token.
- `RSE_NAMESPACE`: Namespace used for replication tests (default: `testing`).
- `RSE_REPLICATION_LIFETIME`: Replication lifetime in seconds (default: `3600`).
- `PUSH_GATEWAY_URL`: Prometheus Pushgateway endpoint.
- `DATA_MANAGEMENT_URL`: Data Management service endpoint.
- `SITE_CAPABILITIES_URL`: Site Capabilities service endpoint.
- `AUTH_URL`: Authentication service API endpoint.
- `IAM_USER_NAME`: Username for IAM authentication.
- `IAM_PASSWORD`: Password for IAM authentication.

## Docker usage

The container uses `etc/docker/init.sh` to prepare the environment and execute the test suite.

- Build the image:
  - `docker build -t ska-src-ms-system-testing-suite:local .`
- Run the tests in the container (using your `.env`):
  - `docker run --rm -it --env-file .env ska-src-ms-system-testing-suite:local`

If you use docker-compose:
- `docker-compose up --build`

Ensure your `.env` includes all necessary endpoints and credentials for the target environment.

## Project structure


- `src/`: Package code and tooling for the suite
- `tests/`: Feature files, flows, and shared test utilities
- `docs/`: Documentation sources and Sphinx configuration
- `etc/`: Development-time configuration and container init resources
- `charts/`: Helm-related content (if deploying test tooling)
- `Dockerfile`, `docker-compose.yml`: Containerization
- `pyproject.toml`, `poetry.lock`: Project and dependency metadata
- `Makefile` and companion `.mk` files: Convenience tasks for dev/CI

## Troubleshooting
- Missing or invalid credentials:
  - Verify `.env` values and environment variables.
- Service endpoints are not reachable:
  - Confirm network access and endpoint URLs.
- Test failures due to environment differences:
  - Align `RSE_NAMESPACE`, lifetimes, and service URLs with your target environment.
  - Run with `-vv` for more logs.