Components

../_images/sdp_components.svg

SDP components and the connections between them.

The diagram shows the components of the SDP system and the connections between them. The components, grouped by their function, are as follows.

Execution Control:

  • The Configuration Database is the central store of configuration information in the SDP. It is the means by which the components communicate with each other and it is implemented using etcd.

  • The Controller Tango device is intended to provide the top-level control of SDP services. Presently, it performs the state transitions in response to commands, reports the statuses of SDP components on the components attribute and updates the healthState based on components’ statuses.

  • The Subarray Tango devices control the processing associated with the SKA subarrays. When a processing block is submitted to SDP through one of the devices, it is added to the configuration database. During the execution of the processing block, the device publishes the status of the processing block through its attributes.

  • The Queue Connector Tango devices provide the interface for exchanging real-time data with other telescope subsystems over Tango. Within the SDP the data travel over the data queues to and from processing deployments.

  • The Console provides a command-line interface to the configuration database to control and monitor the system.

  • The Processing Controller controls the execution of processing blocks. It detects them by monitoring the configuration database. To execute a processing block, it requests the deployment of the corresponding script by creating an entry in the configuration database.

Quality Assessment (QA):

  • The Display Server serves the client with the HTML and JavaScript required to display QA data provided by the API server.

  • The API Server reads QA data from the data queues and serves them to the client to display. Information and statistics regarding the processing blocks are available at REST endpoints, and data for visualisation is provided by WebSocket endpoints.

Data Queues:

  • The Data Queues are the means by which components of the system exchange data in real time. They are implemented using Kafka.

Platform:

  • The Helm Deployer is the service that the platform uses to respond to deployment requests in the configuration database. It makes deployments by installing Helm charts (a collection of files that describe a set of Kubernetes resources) into a Kubernetes cluster.

  • Kubernetes is the underlying mechanism for making dynamic deployments to process data.

Processing Block Deployment:

  • A Processing Script orchestrates the execution of a processing block. It connects to the configuration database to retrieve the parameters of the processing block and to request the creation of processing deployments.

  • A Processing Deployment is created by the script to process data. It can read and write data products in the data product store. It may also exchange data in real time with other components of the system via the data queues, including producing data for the QA display. Specialised deployments are used to receive visibility data from the Correlator Beamformer.

Data Product Store

  • The Data Product store is a filesystem provided by the platform where processing deployments can read and write data products.