SKA telescope developer portal

Welcome to the Square Kilometre Array software documentation portal. Whether you are a developer involved in SKA or you are simply one of our many users, all of our software processes and projects are documented or linked to in this portal.

The portal is frequently updated as the project evolves; if you feel that something is missing, please have a look at our guide to contributing to the developer portal

If you’re new to developing the SKA, please have a look at our Onboarding material and the guideance on setting up your development environment.

Note

Please also read the SKA Code of Conduct, which governs all SKA interactions.

What follows is a brief guide to the headings you’ll find in the left-hand sidebar of this site. Feel free to explore!

Getting Started and the SKA Developer Community

This section is about getting you up and running. It contains the onboarding material for all new SKA developers, the general contribution guidelines when working on SKA projects, guidance on setting up your development environment, and a list of projects, so you know what the SKA is working on. There is also a wealth of information about our development tools and practices, which you can read as you start development work.

Development tools and practices

Here is the journey SKA Developers take to turn ideas into deployed code. Some of these tasks will be done every day; others less frequently. Feel free to click on each item to navigate to relevant explanations, tutorials, how-to guides or reference documentation.

flowchart TD subgraph Getting Started A[Develop Inside a Container] --> C[Setup PyTango] B[Develop Locally] --> C C --> D[Set up Environment: Clone the Repo] C --> E[Set up Environment: Create the Repo] E --> F[Create the Work Branch] D --> F end subgraph Coding & Testing F --> F1[Write Code] F1 --> G[Lint Code] G --> H[Build Python Wheel] H --> S[Get Unit Tests Running Locally] S --> F1 end subgraph Review S --> U[Push Code to Gitlab] U --> V[Create MR] V --> W[Code Review] W --> F1 end subgraph Continuous Integration S --> I[Write Your Dockerfile] I --> J[Build and Run Your Dockerfile] J --> K[Get Tests Running in Cloud/CICD] end subgraph Continuous Deployment K --> L[Create/Update My Helm Chart] L --> M[Integrate with DBs] L --> N[Integrate with Secrets] M --> O[Run Integration Tests] O --> L N --> O end subgraph Release O --> P[Update CHANGELOG/Documentation] P --> Q[Update Versions/Release] Q --> R[Deploy to PSI/ITF?] end

PyTango Developer Journey

Note

This journey is based on a pytango developer journey for now but will be changed to support more later.

Setup Environment

  1. Develop Inside a Container

  1. Setup PyTango

  2. Set up Environment: Clone the Repo

  3. Create the Work Branch

  1. Develop Locally

  1. Setup PyTango

  2. Set up Environment: Clone the Repo

  3. Create the Work Branch

Coding & Testing

  1. Write Code

  1. Lint Code

  1. Build Python Wheel

  2. Get Unit Tests Running Locally

Review

  1. Push Code to Gitlab

  2. Create MR

  3. Code Review

Continuous Integration

  1. Write Your Dockerfile

  1. Build and Run Your Dockerfile

  1. Get Tests Running in Cloud/CICD

Continuous Deployment

  1. Create/Update My Helm Chart

  2. Integrate with databases

  1. Integrate with Secrets

  2. Run Integration Tests

Release

  1. Update CHANGELOG/Documentation

  1. Update Versions/Release

  1. Deploy to PSI/ITF?

SKA git repositories

The SKA uses git as its distributed version control system, and all SKA code shall be hosted in an SKA organisation. The gitlab organization ska-telescope can be found at https://gitlab.com/ska-telescope. All SKA developers must have a gitlab account and be added to the organisation. See Working with GitLab for further details.

Working with SKA Jira

Every team is tracking daily work in a team-based project on our JIRA server at https://jira.skatelescope.org. Our internal wiki, Confluence, has guidance on how we use JIRA. We rely on integrations between GitLab and JIRA to manage our work.

CI/CD: Continuous Integration and Deployment

CI/CD is at the heart of SKA development, and we use GitLab’s automation extensively, so we can test and deploy our software more efficiently.

Testing

Tests are a key part of producing working software. We suggest you look at our Software Testing Policy and Strategy, and our BDD testing guide and BDD Walkthrough.

Test Infrastructure

To support our testing and CI/CD pipelines, we have the multiple kubernetes clusters configured to allow testing to happen.

Containerisation

To facilitate code portability and reliability and test running, we use containers. We also use kubernetes as our container orchestration system.

Working with BinderHub

To enable developers to share their code and collaborate, while being provided with enough computational resources to execute it.

Documentation

While we prefer working code over documentation (as Agile developers), we also recognise that this is a large and long-lived project, so documentation has an important place.

Package Release Process

What you need to know in order to release an SKA software package.

Vault Secret Management

What you need to know in order to keep your secrets secret.

Logging

Making sure your software project outputs useful logs for the SKA

Monitoring Dashboards

You’ve deployed your code on one of our test systems. Now you want to monitor it.

Bug Reporting

What to do when you find a bug in SKA code.

Coding Guidelines

Guidelines to the coding standards we apply in the SKA. Not available for all languages.

FAQ

Questions frequently asked by developers.

Policies and Procedures

Fundamental SKA Software & Hardware Description Language Standards

These standards underpin all SKA software development. The canonical copy is held in eB, but the essential information is here:

Software Product Quality Assurance plan

This Software Product Quality Assurance Plan (SPQAP) defines the Quality Assurance requirements and management activities for the development of software by the Agile Development Teams through the PSSC Contracting arrangements, and the delivery of the software by the SKAO.

This document is a mapping to the SKA Product Quality Assurance (PQA) Plan, that describes the roles and responsibilities of the SKAO and the Contractors and provides an itemised response to the Product Quality Assurance Management, and the PQA Requirements sections.

Software Development Security Policy

Most of the software used for the control and operation of the SKAO telescopes will be developed by SKA Community members and contributors. To ensure the smooth operation of the SKAO and minimise the risk of software vulnerabilities, information security must be incorporated within the software development lifecycle of SKAO software. This document describes how this must be achieved.

Compute Access Policy

While we like to keep as much as possible open, we can’t allow everyone access to the computing hardware that powers our telescopes. Therefore we have defined groups who can get certain kinds of access to our computing hardware, and that is described in this document.

Definition of Done

The definition of done is used to guide teams in planning and estimating the size of stories and features:

Testing Policy and Strategy

The testing policy and strategy is used to align people to the goals of testing and to guide them in establishing an effective testing process:

Incident Management

The incident management workflow is used to guide teams in dealing with anomolous conditions that lead to some form of service outage, unexpected system behaviour or degraded system performance:

About the SKA

For information about the SKA, have a look at this section.

Contribute to the SKA Developer Portal

We encourage all members of the development community to submit improvements to the Developer Portal. These pages describe how you can contribute changes to the Developer Portal.