Changelog

0.20.0

• Update LMC to 0.29.0:

  • Revise controller health state logic: The SDP health state is reported as “DEGRADED” if some, but not all, subarrays are offline; it is downgraded to “FAILED” if all subarrays are offline.

  • Subarray rejects AssignResources command with processing blocks if the SDP is not able to start processing (the processing controller or Helm deployer is offline).

  • Subarray reports health state as “DEGRADED” when processing controller or Helm deployer are offline.

  • Ensure that subarray generates events for transitional obsStates even when they do not correspond to meaningful internal states.

• Update processing controller to 0.13.0. The code has been refactored without changing the functionality.

0.19.1

• Fix SKB-315 by adding ORB endpoint arguments when deploying Tango device servers.

• Update etcd to 3.5.12.

• Update Taranta chart dependency to 2.7.3.

• Remove Taranta auth chart dependency.

0.19.0

• Update LMC to 0.28.0:

  • Update configuration library to 0.5.6 and close connection to configuration DB upon exit.

  • Update PyTango to 9.5.

  • Push change and archive events for the versionId attribute.

  • Implement writeable admin mode attribute.

  • Replace the deprecated “MAINTENANCE” admin mode value with “ENGINEERING”.

  • Remove the Disable command from the controller device.

• Update Helm deployer to 0.14.2:

  • Update configuration library to 0.5.6 and close connection to configuration DB upon exit.

  • Ensure background monitoring threads stay alive and continue running even in the face of unexpected exceptions.

• Update processing controller to 0.12.3:

  • Update configuration library to 0.5.6 and close connection to configuration DB upon exit.

• Update console to 0.5.2:

  • Update configuration library to 0.5.6 to pick up fix for ska-sdp watch command.

• Add options for creating cloned PVCs to share the data product volume between namespaces. This depends on the custom SKAO infrastructure for creating cloned PVCs.

• Add readiness probes for LMC, processing controller and Helm deployer.

• Set environment variable OMP_NUM_THREADS=1 for controller and subarray devices.

• Update LMC queue connector to 4.0.4.

• Update QA chart dependency to 0.24.0.

• Update SKA Tango base chart dependency to 0.4.9.

• Update Taranta chart dependency to 2.6.0.

• Update Taranta auth chart dependency to 0.2.0.

0.18.1

• Add values to set the subarray AssignResources and Configure command timeouts.

• Add Engineering Data Archive (EDA) configuration file.

• Package EDA configuration file and Taranta dashboard in the chart.

• Fix ClusterRole and ClusterRoleBinding setup for Helm deployer to allow multiple SDP deployments to coexist when monitoring for network attachment definitions.

• Remove old network attachment definition configuration setup.

• Update LMC to 0.26.0 to:

  • Enable subarray command timeouts to be configured;

  • Read SDP version information from configuration database.

• Update Taranta chart dependency to 2.5.1.

0.18.0

• Reorganise and update documentation.

• Add global value to specify processing namespace (global.sdp.processingNamespace).

• Update LMC to 0.25.0 to:

  • Add new sdpVersion attribute to controller to publish SDP version information;

  • Rename version attribute to versionId to conform to standard interface;

  • Allow access to device FQDN in the receive_addresses subarray attribute;

  • Allow dependencies between real-time processing blocks;

  • Upgrade PyTango to 9.4.2.

• Update LMC Queue Connector to 3.0.1. This provides support for multiple configurations to run on a single Queue Connector device instance. (MR240)

• Update processing controller to 0.12.2 to

  • Allow dependencies between real-time processing blocks to exist without affecting their scheduling;

  • Fix logic to handle cases where an configuration database entity has vanished.

• Upgrade Helm deployer to 0.41.1 to

  • Fix listing of releases when there are more than 256 installed by using pagination;

  • Add support for tracking network attachment definitions so that the entries in the configuration database can be maintained.

• Update QA chart dependency to 0.23.2.

• Update SKA Tango base chart dependency to 0.4.7.

• Update TangoGQL chart dependency to 1.3.10.

• Update Taranta chart dependency to 2.4.2.

• Add Taranta auth chart 0.1.9 as dependency

• Move Kafka chart 23.0.1 dependency from QA chart to main SDP chart.

0.17.0

• Add initial support for native network discovery in visibility receive.

• Add Taranta 2.3.1 as chart dependency.

• Add TangoGQL 1.3.8 as chart dependency.

• Fix SKB-180 by replacing configuration library etcd backend with a more performant implementation. All components interacting with configuration database has been updated to use the new version, and etcd has been updated to 3.5.9.

• Fix implementation of admin mode attribute on controller and subarray Tango devices.

0.16.1

• Update LMC queue connector to 1.1.1.

0.16.0

• Add “Using the SDP” section in the documentation.

• Add checks on processing namespace and Helm release prefix to avoid problems if the SDP is deployed in a single namespace (not a recommended configuration).

• Update LMC to 0.22.0. This changes the behaviour of subarray Configure command to wait for processing resources to be ready before entering READY observing state.

• Update Helm deployer to 0.12.4.

• Update LMC queue connector to 1.1.0.

• Update QA chart dependency to 0.18.0.

• Update Tango base chart dependency to 0.4.6.

• Remove QA metrics Tango device. This functionality will be exposed via the queue connector Tango devices.

• Remove operator web interface. This was developed as a proof-of-concept method to monitor the configuration database, but it had not been updated for some time and was not exposed to users outside the cluster.

• Remove data product dashboard chart dependency. This was an optional dependency of the SDP, but it will now be deployed as a separate product.

0.15.0

• Add data product dashboard 0.2.3 as chart dependency

• Use global value to specify name of data product PVC (global.data-product-pvc-name)

• Update QA chart dependency to 0.15.1.

• Rename data exchange Tango device as LMC queue connector and deploy one for each subarray.

0.14.1

• Update components

  • LMC 0.21.1 (REL-507)

  • Processing controller 0.11.5 (REL-506)

  • Console 0.4.5 (REL-505)

  • Configuration library 0.4.5 (REL-504)

• Update to tests, including adding a BDD test for monitoring with the controller device and using the visibility receive script 0.8.1

• Fix SKB-205

• Fix SKB-211

0.14.0

• Add Kubernetes liveness probes on the processing controller, Helm deployer, and controller and subarray Tango devices

• Add data exchange Tango device.

• Update components

  • Console 0.4.4

  • Operator interface 0.3.2

  • Helm deployer 0.11.4

  • Processing controller 0.11.4

  • LMC 0.21.0

  • Configuration library 0.4.4

• Fix SKB-190

0.13.1

• Update QA chart dependency to 0.14.0.

0.13.0

• Add options to use an existing persistent volume claim or create a new one for storing data products.

• Update QA chart dependency to 0.13.0.

• QA display is deployed by default.

• Add QA metrics Tango device (not deployed by default).

• Visibility receive test downloads telescope layout using telescope model client.

0.12.4

• Update LMC to 0.20.4. This allows the subarray Tango device to accept the Abort command in RESOURCING.

0.12.3

• Update components

  • LMC 0.20.3 (REL-363)

  • Processing controller 0.11.2 (REL-362)

• Update visibility receive tests to use CBF emulator 4.0.1.

• Fix SKB-179.

0.12.2

• Add QA chart dependency. It includes a display server and an API server. It has a dependency on Kafka for the data queues.

• Add extended test of visibility receive.

0.12.1

• Add options to control configuration database (etcd) data retention.

• Update LMC to 0.20.2. This introduces changes to subarray commands:

  • AssignResources, ReleaseResources and ReleaseAllResources commands cancel execution block if one is already assigned.

  • End command is allowed in IDLE.

• Fix SKB-184.

0.12.0

• Add time-to-live (TTL) to configuration jobs that are run when the SDP is installed. The TTL is set to 30s.

• Update user documentation.

• Update LMC to 0.20.1 to support new subarray command and attribute schemas (0.4) for multi-scan observations.

• Update visibility receive test to run multiple scans and to use the new subarray schemas.

• Update integration tests to use strict schema validation.

0.11.2

• Update console to 0.4.1 with latest version of ska-sdp command-line interface which enables deletion of a large number of configuration database entries simultaneously.

• Update Helm deployer to 0.11.1 to enable monitoring of pods that it deploys.

0.11.1

• Update processing controller to 0.11.1 to enable cleaning up of execution blocks and processing blocks.

0.11.0

• Implement the Great SDP Renaming to bring the internal interfaces into line with names used externally. Processing script (formerly known as workflow) versions need to be updated because there have been breaking changes to the internal interfaces.

• Update LMC to use device names defined in ADR-9 and add option to revert to the old names for backwards compatibility.

• Remove proof-of-concept version of QA display and Kafka chart dependency.

0.10.1

• Update components

  • Operator interface 0.2.3

  • LMC 0.18.3

• Remove Dockerfile and CI job to build OCI image for testing in Skampi pipeline.

0.10.0

• Add option to add prefix to Helm deployer chart releases.

• Update Helm deployer values keywords.

0.9.2

• Add CI job to upload test results to Jira.

• Update implementation of the visibility receive test.

0.9.1

• Change app label to be the same as the chart name (ska-sdp).

0.9.0

• Update components

  • Console 0.3.3

  • Operator interface 0.2.2

  • Processing controller 0.10.2

  • Helm deployer 0.9.3

  • LMC 0.18.2

• Update Tango base chart dependency to 0.3.6.

• Remove option for creating processing namespace.

• Remove deprecated option for creating Helm deployer ClusterRole.

0.8.3

• Add Dockerfile and CI job to build OCI image for running tests in Skampi pipeline.

• Refactor tests to enable them to be run in Skampi pipeline.

0.8.2

• Add proof-of-concept version of QA display and Kafka chart dependency.

• Add tests to simulate subarray Tango device failure.

• Update components

  • LMC 0.18.1

• Update repository to use standard make targets and CI jobs maintained by the System team.

• Remove deprecated option to use etcd-operator.

0.8.1

• Update Console to 0.3.2. This allows CLI to create real-time processing blocks and end scheduling block instances.

0.8.0

• Update LMC to 0.18.0. This improves the behaviour of commands so attributes take their transitional or final value before the command exits.

0.7.0

• Rename chart to ska-sdp.

• Update components to use images from the central artefact repository:

  • LMC 0.17.2

  • Console 0.3.1

  • Operator interface 0.2.1

  • Processing controller 0.10.1

  • Helm deployer 0.9.1

0.6.1

• Update Tango base chart dependency to ska-tango-base 0.2.23.

• Remove time-to-live settings from configuration jobs so they are not deleted on Kubernetes 1.21.

0.6.0

• Update LMC to 0.17.1. This adds support for version 0.3 of the command interface schemas. Support for version 0.2 is retained for backwards compatibility.

0.5.0

• Add operator web interface, version 0.2.0.

• Update console to 0.3.0. This makes the new ska-sdp command-line interface available.

• Update processing controller to 0.10.0. This reads the workflow definitions from the configuration database.

• Add Job to import the workflow definitions into the configuration database.

• Update Helm deployer to 0.9.0. This uses a configuration watcher in the main loop and improves the handling of temporary YAML files.

• Reduce size of component Docker images by using slim Python base image.

0.4.1

• Update Helm deployer to 0.8.0. Values to pass to a Helm chart are now written to a temporary YAML file. This allows hierarchical structures in the values to be passed to the charts.

• Update processing controller to 0.9.0. This is needed to work with the new version of the Helm deployer.

• Update console to 0.2.1. This updates the software versions installed in this component.

0.4.0

• Add values to set the telescope prefix in Tango device names and the number of subarrays to be deployed.

• Update Tango base chart dependency to 0.2.16.

0.3.7

• Update LMC to 0.16.2. This implements ADR-22 (versioning of JSON schemas) in the subarray device. Only the latest version of the interface (0.2) is supported by the AssignResources and Configure commands.

0.3.6

• Update LMC to 0.16.1. This works around a problem in the etcd interface which could block the event loop and prevent it from updating the device attributes.

0.3.5

• Update processing controller to 0.8.0. This resolves the transaction operation limit problem in the main loop.

0.3.4

• Update LMC to 0.16.0. This enables log messages to be written to the Tango logging system in addition to the standard output.

0.3.3

• Update Helm deployer to 0.7.2. This enables it to use a prefix for chart releases. This is intended to enable deployment of the SDP in a single namespace for testing purposes, however this is not recommended.

• Add option for setting the Helm deployer release prefix.

0.3.2

• Update URLs for new repository locations.

0.3.1

• Fix Helm deployer chart repository environment variable.

0.3.0

• Update LMC to 0.15.0. This adds transaction IDs to commands.

• Add option to enable passing a transaction ID to all commands. By default it is only enabled on commands that previously accepted arguments, to avoid changing the argument type on the other commands.

0.2.1

• Update LMC to 0.14.2. This correctly configures the Tango device attributes to use change events. This also stores the master device state in the configuration database.

0.2.0

• Remove dependency on the etcd-operator chart. Add option to enable the use of etcd-operator if it is available.

• Add option to set the etcd transaction operation limit. This is not supported by etcd-operator.

• Disable the Helm deployer ClusterRole (used to manage persistent volumes) by default. Add option to re-enable it if needed.

• Update Tango base chart dependency to 0.2.8.

0.1.0

• Initial release of sdp chart.