Configure your development environment#

Configure your development environment

Set up your local machine for SKAO development. Choose container-based development (recommended) or local installation.

What you’ll learn#

By the end of this tutorial, you’ll have:

  • Installed the base tools: Git, Python, a Python package manager, and Docker

  • Configured Git and GitLab for SKAO contributions

  • Opened a repository in its devcontainer

  • Deployed a local Minikube cluster and confirmed it is healthy

  • Connected to an SKAO CI/CD VPN and resolved internal domains

Time required: 1–2 hours for the local setup. VPN access depends on a support request, so start that first if you need it.

Prerequisites:

  • Ubuntu, either installed directly or running under WSL. The Minikube section also supports macOS on Apple Silicon

  • An SKAO account for the GitLab and support-request steps

  • Administrator rights on your machine to install packages

Choose your setup path#

Your development environment depends on the type of work you do. Container-based development is recommended when possible, but you may need to develop locally or remotely over SSH.

Two main approaches:

  1. Local development — Install tools directly on Ubuntu, or on Ubuntu under WSL

  2. Container development — Use Docker for isolated, reproducible environments (recommended)

Set up local development#

Install base system packages#

These steps assume Ubuntu, either installed directly or running under WSL. Install python3 venv, Git, and build tools:

sudo apt install git python3-venv build-essential

Configure Git and GitLab#

SKAO requires your institutional email, two-factor authentication, and GPG-signed commits. Follow Tutorial: Make your first contribution to configure Git, add an SSH key, and enable 2FA, then use How to work with GitLab at SKAO for the individual tasks and GitLab reference for the branch and commit formats.

Set Python 3 as default#

sudo apt install python-is-python3

Install a Python package manager#

SKAO is migrating its projects to uv. Install uv for new work:

curl -LsSf https://astral.sh/uv/install.sh | sh

Many SKAO repositories still use Poetry. Install it when you work on one of them:

curl -sSL https://install.python-poetry.org | python3 -

Check each repository’s pyproject.toml to see which manager it uses.

Install Docker#

Follow the Docker Engine installation guide for Ubuntu. Complete the Linux post-installation steps to run Docker without sudo.

Under WSL, follow the same guide to install Docker Engine natively inside the Ubuntu distribution. Do not use Docker Desktop.

Configure VS Code#

VS Code is the recommended IDE. Install these required extensions:

Recommended extensions:

  • GitLens

  • Jupyter

  • Makefile Tools

  • YAML

Note

PyCharm is also used by SKAO developers, but some features for remote and container development require the paid version.

Set up container development#

Container-based development keeps dependencies and tool versions inside the project instead of on your machine. Every developer on a repository then works from the same toolchain, and that toolchain matches the one the CI pipeline uses.

A development container, or devcontainer, describes that environment in a .devcontainer/devcontainer.json file committed alongside the code. VS Code, and any other editor that implements the specification, reads the file and builds the container for you.

Prerequisites#

Open a repository in its devcontainer#

Clone the repository and open it in VS Code:

git clone <repository-url>
cd <repository> && code .

When the repository defines a devcontainer, VS Code offers to reopen the folder in the container. Accept the prompt, or run Dev Containers: Reopen in Container from the command palette. VS Code builds the image on first use and reuses it afterwards. The terminal, extensions, and debugger then run inside the container while your files stay on the host.

Add a devcontainer to a repository#

A repository without a .devcontainer/ directory needs one before this workflow applies. Run Dev Containers: Add Dev Container Configuration Files in VS Code to generate a starting point, then commit the result so the rest of the team gets the same environment.

Consult the devcontainer specification for the available properties and the VS Code Dev Containers documentation for the editor workflow.

Note

Not every SKAO repository ships a devcontainer. Check the repository’s own README for the supported development workflow before assuming this one applies.

Set up a local Kubernetes environment#

Deploy a local Minikube cluster to develop and test charts before you deploy them to a shared SKAO cluster. Use ska-cicd-deploy-minikube, which installs the tools and deploys a cluster configured to emulate the SKAO testing and production environments.

The repository supports Ubuntu, macOS on Apple Silicon, and Ubuntu under WSL. It installs Minikube, kubectl, helm, k9s, and yq, and enables the metrics-server and ingress addons.

Prepare the prerequisites#

Install Docker as described above and add your user to the docker group:

sudo usermod -aG docker $USER

Install make, curl, unzip, and git. The deployment installs its tools into ~/.local/bin, so put that directory on your PATH:

export PATH="$HOME/.local/bin:$PATH"

Add that line to ~/.bashrc or ~/.zshrc to make it permanent.

Deploy the cluster#

Clone the repository and deploy:

git clone https://gitlab.com/ska-telescope/sdi/ska-cicd-deploy-minikube.git
cd ska-cicd-deploy-minikube
make all

make all installs the command-line tools and then deploys the cluster. Run make on its own to list the available targets and variables, and make minikube-vars to show the current settings and cluster status.

Verify the deployment#

make platform-info
make verify
kubectl get nodes

make platform-info reports the detected operating system, driver, and PATH. make verify confirms that the tools are installed and the cluster is healthy. The node must report Ready.

Reach services in the cluster#

On Ubuntu, the deployment configures HAProxy and automatic DNS. On macOS and WSL these are not deployed, so reach services through minikube ip or kubectl port-forward instead:

minikube --profile minikube ip

Under WSL, the Docker internal address is not reachable from the Windows host. Use the WSL address instead:

hostname -I | awk '{print $1}'

Clean up#

Delete the cluster when you finish:

make clean

Warning

make clean removes the Minikube cluster and everything deployed in it. There is no undo.

Consult the repository README for the Tango operator, image preloading, Vault and Flux addons, and the current platform-specific notes.

Check the PyTango documentation for more information.

Connect to the SKAO CI/CD VPNs#

Connect to the VPN for the platform that hosts the environment you need to use. TechOps and DP use separate VPN profiles, but you request and connect to both the same way.

Choose the VPN#

Platform

Datacentre identifier

Use it for

TechOps

stfc-techops

Shared CI/CD deployments and services hosted on the STFC TechOps cluster.

DP

stfc-dp

SDP integration and data-processing deployments hosted on the STFC DP cluster.

Warning

A TechOps VPN profile does not provide access to DP. Request the profile for the platform you need, and never share a VPN profile or commit it to a repository.

Request VPN access#

  1. Open a VPN access request and request access to the VPN you need.

  2. State the platform, TechOps or DP, and the reasons why you need VPN access.

  3. Follow the secure profile-delivery instructions from the System Team.

Note

The support request requires you to sign in to the SKAO Jira service desk.

You can also request VPN access to production environments. Ask in the #team-bang Slack channel instead of opening a System Team support request.

Connect to the VPN#

Install a supported OpenVPN client and import the .ovpn profile supplied by the System Team. On a Linux system with the OpenVPN command-line client, run:

sudo openvpn --config /path/to/your-profile.ovpn

Keep that terminal open while you use the VPN. For a desktop client, import the same profile and activate the imported connection.

After connecting, open the service URL supplied by the deployment pipeline or the service documentation. Disconnect the VPN when you finish.

Resolve internal domains on WSL#

These steps cover running the OpenVPN client on the Windows host while you work inside a WSL distribution. The VPN configures DNS for Windows, but the WSL distribution does not inherit that configuration, so internal SKAO domains fail to resolve inside WSL even while the VPN is connected. Run dnsmasq inside WSL to forward internal domains to the internal DNS servers and everything else to your usual upstream resolvers.

Note

Running the OpenVPN client inside the WSL distribution itself is a different case that this tutorial does not cover.

Install dnsmasq before you change the resolver configuration, while name resolution still works:

sudo apt install dnsmasq

Add the listener and the internal domain to /etc/dnsmasq.conf:

listen-address=127.0.53.1
bind-interfaces
port=53

server=8.8.8.8
server=1.1.1.1

server=/techops.internal.skao.int/192.168.99.162

The 127.0.53.1 address keeps dnsmasq clear of 127.0.0.53, which systemd-resolved binds on some distributions.

Stop WSL from regenerating the resolver configuration, and enable systemd so that dnsmasq starts as a service. Add this to /etc/wsl.conf:

[network]
generateResolvConf = false

[boot]
systemd=true

Point the distribution at dnsmasq. Replace the contents of /etc/resolv.conf with:

nameserver 127.0.53.1

Restart WSL from a Windows terminal so that both files take effect:

wsl --shutdown

Connect the VPN, reopen the distribution, and confirm that an internal name resolves:

nslookup <host>.techops.internal.skao.int

Note

A separate internal domain is not available for the DP cluster.

Was this page helpful?