Deployment to Kubernetes
The ODA is deployed into a Kubernetes cluster when running in the CICD pipeline environments, and can also be deployed locally.
For more info on how to deploy locally, and how to access the CICD environments, see the README.
The options below TODO
Backend Persistence Options
As described in the other sections, the ODA is implemented with generic interfaces which are implemented using different persistence mechanisms - memory, filesystem and postgres.
The REST server can be configured at deploy time to use any of these via the ODA_BACKEND_TYPE environment variable. This in turn can be set in the Helm chart from the values.yaml.
Note
These instruction assume you are using the standard SKA Minikube installation that is configured and installed via the ska-cicd-deploy-minikube project.
Filesystem
Using the filesystem backend will persist data to the filesystem of the ska-db-oda application. A Kubernetes Persistent Volume ensures
data survives Kubernetes redeployments and pod restarts. To configure, set the following values for the ska-db-oda Helm chart:
rest:
...
backend:
type: filesystem
filesystem:
# true to mount persistent volume, false for non-persistent storage
use_pv: true
# path on Kubernetes host to use for entity storage
pv_hostpath: /mnt/ska-db-oda-persistent-storage
...
For a default installation with no Helm value overrides, access the PersistentVolume on the minikube node as follows:
$ # SSH to minikube cluster
$ minikube ssh
$ # navigate to default ODA storage directory
$ cd /mnt/ska-db-oda-persistent-storage/
Files can also be stored outside minikube by making pv_hostpath match the
MOUNT_FROM and MOUNT_TO values set when rebuilding minikube with the
ska-cicd-deploy-minikube project. For example, if minikube is rebuilt with
$ make MOUNT_FROM=$HOME/oda MOUNT_TO=$HOME/oda all
and pv_hostpath is set to match $MOUNT_TO, entities will be stored
directly on your local filesystem in the $HOME/oda directory. See the
bottom of this page for a full example.
Postgres
Using the postgres backend will persist data to an instance of PostgreSQL.
To use postgres as a backend, a running instance of PostgreSQL must be available, and the Helm values set as below:
rest:
...
backend:
type: postgres
postgres:
host: 12.34.56.8
port: 5432
user: username
password: password
db:
name: "postgres"
...
By default, the make k8s-install-chart target will set the postgres values to those required to connect to the PostgreSQL instance also deployed by the chart.
To access the postgres instance directly (for example using psql as described in PostgreSQL), the IP address of the LoadBalancer can be found with kubectl get svc -n ska-db-oda | grep LoadBalancer | awk '{print $4}'.
Note
If using Minikube, the LoadBalancer needs to be exposed via minikube tunnel See here for more details.
Memory
Using the memory backend will store data in the memory in the Python objects. This is mainly used as a lightweight test environment for applications that have an ODA dependency.
To configure, set the following values for the ska-db-oda Helm chart:
rest:
...
backend:
type: memory
...
Example: ODA deployment using a local directory for backend storage
This example is useful for local testing, and for testing application which use the ODA without having to think about postgres.
Caution
This will delete any existing Minikube deployment!
In this example, the user tango wants to deploy the ODA so that the ODA
stores and retrieves SBs from the local directory /home/tango/oda. We’ll
set an environment variable to hold this location.
$ export ODA_DIR=$HOME/oda
Minikube needs to be deployed with a persistent volume that makes $ODA_DIR available inside the Kubernetes cluster. This is achieved by redeploying Kubernetes using the ska-cicd-deploy-minikube project. Checkout the project and (re)deploy Minikube like so:
$ # checkout the ska-cicd-deploy-minikube project
$ git clone --recursive https://gitlab.com/ska-telescope/sdi/ska-cicd-deploy-minikube
$ cd ska-cicd-deploy-minikube
$ # redeploy Minikube. Caution! This will delete any existing deployment!
$ make MOUNT_FROM=$ODA_DIR MOUNT_TO=$ODA_DIR clean all
The ODA chart can now be installed. For a local installation, it can be useful
to expose the ODA ingress so that the ODA deployment can be exercised from
outside Minikube, i.e., from your host machine. We also want to configure the
ODA to use the directory exposed at $ODA_DIR. These aspects are configured
by setting the relevant Helm chart values. These values could be set
individually using K8S_CHART_PARAMS="--set parameter1=foo --set
parameter2=bar" etc., but as there are several values to set we will define
them in a setting file (overrides.yaml) to be included when deploying the ODA.
$ # navigate to the directory containing the ska-db-oda project
$ cd path/to/ska-db-oda
$ # inspect contents of our Heml chart overrides. This example enables ODA
$ # ingress and configures the backend to to use the persistent volume
$ # exposed at $ODA_DIR. Create this file if required.
$ cat overrides.yaml
rest:
ingress:
enabled: true
backend:
type: filesystem
filesystem:
use_pv: true
pv_hostpath: /home/tango/oda <-- replace with the value of $ODA_DIR
$ # install the ODA including the custom values
$ make K8S_CHART_PARAMS="--values overrides.yaml" k8s-install-chart
The state of the deployment can be inspected with make k8s-watch. The
output for a successful deployment should look similar to below:
$ make k8s-watch
NAME READY STATUS RESTARTS AGE
pod/ska-db-oda-rest-test-0 1/1 Running 0 24s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ska-db-oda-rest-test ClusterIP 10.98.75.197 <none> 5000/TCP 24s
NAME READY AGE
statefulset.apps/ska-db-oda-rest-test 1/1 24s
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pvc-b44332d8-e8b8-472b-a407-2080c850dee0 1Gi RWO Delete Bound ska-db-oda/ska-db-oda-persistent-volume-claim-test standard 24s
persistentvolume/ska-db-oda-persistent-volume-test 1Gi RWO Delete Available standard 24s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/ska-db-oda-persistent-volume-claim-test Bound pvc-b44332d8-e8b8-472b-a407-2080c850dee0 1Gi RWO standard 24s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/ska-db-oda-rest-test <none> * 80 24s
SBs uploaded to the ODA will be stored in $ODA_DIR. Any SB JSON files
stored in $ODA_DIR directory will be retrievable via the ODA REST API, e.g.,
$ # get the ODA endpoint, which is a combination of Minikube IP address and
$ # deployment namespace
$ MINIKUBE_IP=`minikube ip`
$ ODA_NAMESPACE=`make k8s-vars | grep 'Selected Namespace' | awk '{ print $3 }'`
$ ODA_ENDPOINT=http://$MINIKUBE_IP/$ODA_NAMESPACE/oda/api/<MAJOR_VERSION>/sbds
$ # get the SBD ID from the SB used for unit tests. We'll upload the SB to this URL.
$ SBD_ID=`grep sbd_id tests/unit/testfile_sample_low_sb.json \
| awk '{ gsub("\"", ""); gsub(",", ""); print $2 }'`
$ # upload the unit test SB to the ODA
$ curl -iX PUT -H "Content-Type: application/json" \
-d @tests/unit/testfile_sample_low_sb.json \
$ODA_ENDPOINT/$SBD_ID
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
Date: Mon, 21 Feb 2022 11:24:25 GMT
Content-Type: application/json
Content-Length: 76
Connection: keep-alive
{"message":"Created. A new SB definition with UID sbi-mvp01-20200325-00001."}
$ # list contents of $ODA_DIR. The uploaded SB should be stored there.
$ ls $ODA_DIR
sbi-mvp01-20200325-00001.json