Changelog

Development

[Remove] remaining refrences and use of additional_windows parameter and high resolution windows code.

[Remove] Calls to telmodel for RFI mask that we don’t use.
[New] Websocket endpoint ‘/ws/earliest/…’ to get earliest messages from Kafka queue. (PHX-256)
[Remove] All Redis access.
[Remove] Many endpoints that the current Display doesn’t use.

[Change] Grafana log links contain cluster info set by the CLUSTER env var.
[Remove] The QA Helm chart has been removed, as it is now built into the SDP chart.
[Update] Move processing of pointing-offset data to websocket.
[Update] Use pointing offset’s Flow entry for the websocket.

[Fix] SKB-914 waterfall plots stopping after a time. redis.zrange result was being incorrectly reversed.
[Change] Update execution block endpoints to incorporate all the necessary information
[Deprecate] Deprecate the execution block receptors endpoint
[Update] Upgraded ska-sdp-config to 1.2.0

[Change] Updated the documentation (PHX-106)
[Change] Migrated Vault config to use new method of fetching secrets (PHX-35)
[Add] New endpoint to retrieve the list of available flow entries
[Change] Only allow websockets to connect if there is a flow entry that is valid
[Change] Update base docker images to latest (0.1.3 & 0.1.4)

[Change] Docker container no longer runs as root.
[Refactor] Refactor the method of fetching flows, and fix the pointing offset topic allocation
[Change] Catch redis exception in the cleanup script on startup
[Refactor] Migrate spectrograms plots data source to phase topic
[Change] Waterfall plots are no longer generated on the API, the source data is sent to requester
[Change] All data entries now use flow entries
[Change] Move stats from fetch requests, to streamed data via websockets

[New] Added the dependency on the current_user_profile onto the create_window endpoint.
[New] Updated the charts to integrate user authentication into the Signal Display.
[New] Create new endpoints to view and create high resolution windows.
[Update] Update to using 0.10.0 of the SDP Config DB library.
[Update] Switch to using the new SKA Python Docker image as a base.
[Removal] The generator/processor is no longer included in the Helm chart.
[Add] Update URL for the data websocket to add partition selection
[Change] Update the crons to select the first partition for all data
[Change] Update API Service in the chart to use port 80
[New] Linked the user email through from the create_window to the configs _windows_create_new_state method.
[Changed] Redis cleanup now also cleans waterfall plots as time passes.
[Update] new Kafka Producer/Consumer that cleans up access to partitions.
[Removal] Crons will no longer keep trying to connect to Kafka, instead opting to use the heartbeat process to try again, this will try again each minute.
[Fix] Correct unpacking of the pointing offset data

[New] [Temp] Add new command for websocket to view current headers
[New] Create new watcher in the Config DB
[Change] All crons now use the new watcher to check for flows
[Breaking] [Change] Topic names (and websocket names) have changed to no longer contain underscores (for ones using the flow entries)
[New] Config Websocket can now get and watch list of flows
[Deprecation] The /stats endpoints should be avoided when querying configuration related data. And in a later version will be removed.
[Change] Removal of our custom Producer and Consumer, and switch to using the DataQueueMultiTopic in the ska_sdp_dataqueues module.
[New] Websocket stats are now saved on connect and disconnect. And last transmit time is saved.

[New] Band average cross correlation websocket added to the config
[Change] Waterfall data now contains timestamps
[New] New helper API endpoint to get Current Subarray Configuration
[Change] Migrate data structures to ska-sdp-dataqueues library
[New] Spectrum data now is in waterfall format
[Change] Improve speed of thumbnail creation for waterfalls
[New] Initial endpoint for RFI Masks
[New] New websocket endpoint that can request configuration, and listen for Config DB changes
[Change] [Breaking] The /consumer endpoint no longer exists. Update to /ws

[Change] Reorder Helm Chart parameters
[Change][Breaking] Cleaning Helm Chart, by moving parameters to new sections to make it easier to see what goes with what
[New] Add gain calibration backend
[New] Add Helm Chart Values validation via a schema
[New] Read configuration from the SDP Config DB, used for endpoints, and for connecting to relevant Kafka topics.
[Change] Update default datatype from json to msgpack`
[Docs] Update documentation to latest

[Fix] Update for datasets with less than 500 channels wide
[New] Add Waterfall limit values to Helm Chart
[New] Read calibration data into new endpoint
[New] Allow overriding the Monitoring links in the chart
[Change] Update method for waterfall plot generation to only save the thumbnails and to stream the full size image
[Deprecate] Disable the Visibility Receive task
[New] Read the stats getting from the generators directly (also uses multiple scripts/processes to do this)