Documenting a project

Documenting a project is the key to make it usable, understandable and maintainable, making the difference between a technically excellent software and a successful software. SKA has defined a set of standards and practices that shall be implemented when developing software documentation. A more comprehensive set of resources can be found online at:

What to document

Based on the documentation provided everyone should be able to:

  • understand the problem the project is trying to solve

  • understand how the project is implemented

  • know exactly what this projects depends on

  • download and build the project

  • execute the project tests

  • run the project

In order to achieve this goal any structure can be used allowing free text as well as code-extracted documentation.

Where to document

There are several places where it’s possible to write SKA documentation: on the developer portal, as part of the Solution Intent, as part of the code repository using sphinx (as described below), in your Team space in Confluence (if you’re a Confluence user), or using Google Docs in SKA’s Google Drive.

Individual code repository documentation is good for describing the current architecture of the code within that repo, and any public APIs it may have. It’s also good for user documentation, deployment information and similar documentation that’s related to the code. It’s not a good place to document information that contains secrets.

The Solution Intent Confluence space is good for planned architecture, developing and modifying the architecture. As the design is developed, some of the Confluence material will migrate into the relevant repositories, or may cause an update to the formal architecture documents via the SKA developer community decision making process. The Solution Intent describes what we’d like to develop; repositories document the code “as-is”, i.e. what is currently implemented in the code. Because Confluence moves ahead of the code, it’s not a good home for documenting the current implementation, especially at the repository level. It is a good space for developing specifications, and sketching out system tests, which eventually may be formulated as BDD tests </tools/bdd-test-context>. The Solution Intent Confluence space is also not always a good home for user documentation, because again it’s hard to know if that’s in sync with the code. It’s clear what should not go in this space: anything that directly describes developed code. The Solution Intent Confluence space is reviewed regularly by the architecture group. If you are not sure where to place documents in the Solution Intent space, the “work” sub-pages for a product are a good place to start. See, for example, the Simulation Work pages. If you can’t find a suitable area, ask on the Architecture Community of Practice Slack channel.

Google Docs is good for drafting formal documents collaboratively, especially documents that will be managed using the SKA Configuration Management system, such as ICDs. If user manuals are to be managed via eB rather than in a repository, then Google Docs is a good tool to use for this. If it is preferred, Overleaf can also be used to manage formal documents collaboratively.

The Developer Portal is best used for SKA software policies and procedures. It is ideally suited for any development-related information that needs to be public, and is applicable to a wide audience. It should not contain information that’s specific to one repo (apart from its own contribution guidelines!) It is not suitable for storing information that needs to be kept private (such as private GitLab variables, passwords, etc.) It isn’t great for information that changes rapidly; we suggest either using other documentation tools until the pace of change is lower, or automating the updates.

Team spaces are good for time-limited material, drafting material for discussion ahead of adding it to a repository, information that needs to be kepts slighly private (i.e. material that can be circulated within SKA, but not to people outside the project). Architecture work is generally better suited to the Solution Intent, where the work can be reviewed and discussed before implementation. For anything that needs reference beyond Program Ingrement (PI) boundaries, some other space may be better, preferably either the Solution Intent or a repository, so that it’s easy to find, and to reduce documentation fragmentation. It’s also less accessible than the developer portal; not all collaborators may have an SKA Community Confluence account. Team spaces are really a private “scratch” space, for communication within the team.

Documenting the Public API

When it comes to software systems, such as services or libraries, it is of paramount importance that the public API exposed by the software component is clearly captured and documented.

How to document

Documentation on git

Each software repository shall contain its documentation as part of the code included in a git repository. In this way the documentation will be released with the same cadence as the code and it will always be possible to trace a particular version of the documentation to a particular version of the code documented.

Free text documentation must be placed in a docs folder in the upper level of the repository structure. Sphinx generates automatically extracted files under this folder as well, wherever there are docstrings in the code.

Using sphinx

Documentation must be realised using the sphinx package and Restructured Text . SKA provides a predefined sphinx template for this purpose in the SKA Python skeleton project. Every project shall use the same docs folder as a starting point for assembling their own documentation.

Sphinx can be used to generate text documents such as this portal, but it also provides capabilities to automatically extract and parse code documentation, or docstrings. Refer to the Python Coding Guidelines for more information.

Extracting documentation from code

Todo

  • add hello world class with parameters to the SKA Python Skeleton Project

  • add code snippet here as an example of additional documentation which is decoupled from code, and describe the pitfalls of separating documentation from the code.

Integration into the Developer Portal

The developer portal is hosted on ReadTheDocs. On the List of projects and subgroups page a list of all the projects that are hosted on GitLab is available, with badges to show the build status of the project’s documentation. Each badge is also a hyperlink to the project’s documentation that you can click on.

Every SKA project’s documentation is hosted on Readthedocs as a subproject of the developer portal, so that all projects have a common URL for easier search-ability. For example: whereas the developer portal’s URL is https://developer.skao.int, the ska_python_skeleton project is at https://developer.skao.int/projects/ska-python-skeleton.

In order to add the project’s documentation as a subproject on ReadTheDocs, a project must first be imported into Readthedocs.

Register on ReadTheDocs

Developers working on the SKA are members of the ska-telescope organisation on GitLab. Registering an account using the OAuth credentials on ReadTheDocs is recommended, because then the integration between the SKA GitLab and SKA ReadTheDocs services is done automatically. The integrations can also be set up manually later, and is not difficult.

Sign up / sign in with GitLab account
Login on Readthedocs

Import project to ReadTheDocs

After signing in, one lands on the Dashboard. To import a project so that it is automatically built by Readthedocs, click on the dropdown next to your name at the top right and choose My Projects, then Import a Project:

My Projects >> Import Project

Find the project you want to import. You can filter the projects using the list on the right, or if you cannot find your project, import manually.

Import a Project
Name the project correctly

When importing the project, the Name/Project Slug should have SKA Telescope (or ska-telescope-, if the project name is in this format) as a prefix followed by the gitlab repository slug. i.e. for ska-tango-examples gitlab repository, the rtd project name should be ska-telescope-ska-tango-examples when being created. This ensures that RTD project slug is correctly set. The RTD project name is used to create the RTD project slug and the slug cannot be changed later whereas the project name can be changed in the settings to “SKA Tango Examples” for readability. If your git repo uses the main branch instead of master, you must fill in the field Default branch: as it is not detected automatically. If there’s an error it’s best to delete the project and recreate it again.

Import a Project
Add a special account as Maintainer

Importantly, the user kurtcobain-19 must be added to the project as a Maintainer so that the System Team can add it as a Sub-project (see next step).

Import a Project
Add project as a sub-project on ReadTheDocs

A sub-project must be added by a user with Maintainer privileges on the main project.

Currently only the System Team members have these permissions. Please ask on the Slack channel #team-system-support to have your project added.

For more information on how to add a subproject, go to Read The Docs.