MID.CSP LMC Documentation

The top-level software components provided by Mid.CSP LMC API are:

Components listed above are implemented as TANGO devices, i.e. classes that implement standard TANGO API. The CSP.LMC TANGO devices are based on the standard SKA1 TANGO Element Devices provided via the SKA Base Classes.

Mid.CSP LMC Controller

Mid.CSP Controller is the top‐level TANGO Device and the primary point of contact for monitor and control of the Mid.CSP Sub-syste.

The Mid.CSP Controller represents Mid.CSP Sub-system as a unit for control and monitoring for general operations.

Mid.CSP Controller main roles are:

  • To be the central control node for Mid.CSP. The Controller provides a single point of access for control of the Mid.CSP as a whole, this includes provision for housekeeping and supervisory commands including: power‐up, power‐down, power management, restart (re‐initialize), support for firmware and software upgrades, etc.

  • To provide rolled‐up reporting for the overall Mid.CSP status. Mid.CSP Controller monitors and intelligently rolls‐up status reported by Mid.CSP equipment, sub‐arrays and capabilities and maintains a standard set of states and modes, as defined in the document “SKA1 Control System Guidelines”. State transitions are reported using standard TANGO mechanism.

  • To implement a set of attributes that represent the status and configuration of the Mid.CSP as a whole and, where required, report availability, status and configuration of the Mid.CSP equipment, components and capabilities (in the form of lists or tables or JSON string).

  • To maintains the pool of resources (VCCs, FSPs, Search/Timing/VLBI beams), keep track of allocation to sub‐arrays and provide reports on resource availability, allocation, and more.

Mid.CSP Controller implementation is based on (derived from) the standard SKA1 TANGO Device Controller; Mid.CSP Controller implements the standard TANGO API, aligned with the document “SKA1 Control System Guidelines”.

The interface is between a TANGO client and a TANGO Device. The TANGO Device exposes attributes and commands to clients.

The roles of the interfacing systems are:

  • TANGO Clients: Mid.TMC sub-system TANGO client(s).

  • TANGO Device: Mid.CSP sub-system implements Mid.CSP Controller.

The clients use requests to obtain read and/or write access to TANGO device attributes, and to invoke TANGO device commands.

Mid.CSP Controller TANGO Device name

The Mid.CSP Controller TANGO Device name is defined in the document “SKA1 TANGO Naming Conventions”:

mid-csp/control/0

Mid.CSP Controller TANGO Device Properties

The Mid.CSP Controller device has a standard set of properties inherited from the SKA Controller TANGO Device and a number of specific properties documented in the Controller API section.

Mid.CSP Controller TANGO Device States and Modes

Mid.CSP Controller implements the standard set of state and mode attributes defined by the SKA Control Model.

Mid.CSP Controller reports on behalf of the Mid.CSP Sub-system – unless explicitly stated otherwise, the state and mode attributes implemented by the Mid.CSP Controller represent the status of the Mid.CSP as a whole, not the status of the Mid.CSP Controller itself.

Mid.CSP Controller operational state

The Mid.CSP Controller supports the following sub-set of the TANGO Device states:

  • UNKNOWN: Mid.CSP is unresponsive, e.g. due to communication loss. This state cannot be reported by CSP itself.

  • OFF: power is disconnected. This state cannot be reported by CSP itself

  • INIT: Initialization of the monitor and control network,equipment and functionality is being performed. During initialization commands that request state transition to OFF (power-down) or re‐start initialization are accepted.

  • DISABLE: Mid.CSP is administratively disabled, either by setting adminMode=OFFLINE or NOT-FITTED. Basic monitor and control functionality is available but signal processing functionality and related commands are not available. All sub‐arrays are empty (OFF) and IDLE; all resources (receptors, tide‐array beams) are placed in the pool of unused resources.)

  • STANDBY: Low‐power state, Mid.CSP uses < 5% of nominal power. Basic monitor and control functionality is available, including the commands to request state transition to ON, OFF, DISABLE, or INIT. Signal processing functionality and related commands are not available. All sub‐arrays are empty (OFF) and IDLE; all resources (receptors, tide‐array beams) are placed in the pool of unused resources.).

  • ON: At least a minimum of CSP signal processing capability is available; at least one receptor and one sub‐array can be used for observing (either for scientific observations or for testing andmaintenance). Mid.CSP is in normal operational state, all commands, including commands to increase/decrease functional availability and power consumption are available.

  • ALARM: Quality Factor for at least one attribute crossed the ALARM threshold. Part of Mid.CSP functionality may be unavailable.

  • FAULT: Unrecoverable fault has been detected, Mid.CSP is not available for use at all, maintainer/operator intervention is required in order to return to ON, STANDBY, or DISABLE. Depending on the extent of failure commands restart and init, as well as status reporting may be available.

Mid.CSP Controller TANGO Device Commands

Mid.CSP Controller implements the standard set of commands as specified in:

  • Standard set of TANGO Device commands as defined in TANGO User Manual

  • Standard set of SKA TANGO Device commands

  • Command specific to Mid.CSP Controller as described in API section.

Mid.CSP makes provision for TM to request state transitions for individual sub‐systems and/or Capabilities.

Mid.CSP Controller TANGO Device Attributes

Mid.CSP Controller implements the standard set of attributes as specified in:

  • Standard set of TANGO Device attributes as defined in TANGO User Manual

  • The standard set of SKA TANGO Device attributes as defined for the SKA Controller TANGO Device.

  • Attributes to Mid.CSP Controller as described in API section.

The Mid.CSP Controller maintains the ‘pool of resources’ and is able to provide information regarding sub‐array membership, status and usage.

Mid.CSP LMC Capabilities

The Mid CSP.LMC Capabilities provide a layer of abstraction to allow the TM to set, control and monitor signal processing, and other sub-system functions, without being aware of the details of the Mid CSP implementation.

Each Mid CSP.LMC Capability represents a subset of the Mid.CSP functionality and uses a subset of the Mid.CSP hardware and software resources.

  • VCCs (Very Coarse Channelizer): 197

  • FSPs (Frequency Slice Processor): 27

  • PSS (Pulsar Search Beams): 1500.

  • PST (Pulsar Timing Beams): 16

The Capabilities implemented by the Mid.CSP can be classified in two groups, as follows:

  • The Capabilities inherent to the sub-arrays or other Capabilities; such Capabilities are created during initialization and are permanently assigned to a particular sub-array or Capability. A Capability can be enabled/disabled, but cannot be removed and/or assigned to another subarray.

  • The Capabilities that are function of resources: these are created during initialization and assigned to a pool of unused resources. Individual instances of these resources may be assigned to subarrays, as commanded by the TM. In some cases the same instance may be used in a shared manner by more than one subarray.

Each Mid.CSP Subarray has the following inherent Capabilities that correspond to Processing Modes, and are configured via scan configuration:

  1. Correlation,

  2. PSS,

  3. PST,

  4. VLBI (beamforming and correlation in support of VLBI), and

  5. Transient Data Capture.

The Capabilities listed above are configured and controlled via a subarray; when a subarray does not perform correlation, its Capability Correlation is OFF; the same applies for all Processing Modes.

The top-level Mid.CSP Capabilities associated to schedulable resources are reported in the table below:

Capability Name

Resources

Notes

Frequency Observing Band

197 VCCs

(Very Coarse Channeliser)

Support frequency bands:

  • Band 1

  • Band 2

  • Band 3

  • Band 4

  • Band 5a

  • Band 5b

FSP Processing Modes

27 FSPs

(Frequency Slice Processor) These resources are shareable among the subarrays

Supported processing modes:

  • correlation

  • pulsar search beamforming

  • pulsar timing beamforming

PSS Processing Mode

1500 Search Beams

Collects the HW, SW and processing modes of PSS: beams:

This capability is a func of the following resources:

  • receptors

  • CBF beamformers

  • PSS pipelines

  • SDP links

PST Processing Mode

16 Timing Beams

Collects the HW, SW and processing modes of PST: beams:

This capability is a func of the following resources:

  • receptors

  • CBF beamformers

  • PST processors

  • SDP links

The previous list does not include VLBI. VLBI beamforming is configured per subarray per FSP, any further beam processing is performed by non-SKA1 equipment; there are no VLBI related Mid.CSP resources that have to be managed, other than the FSPs.

Mid.CSP LMC Subarray

The Mid.CSP Sub-array TANGO Device class is required as a common interface feature, as described in the document “SKA1 Control System Guidelines”.

Its purpose is to provide a point of access for configuration, execution and monitoring of signal processing functionality for each sub‐array independently.

Overview of the functionality

  1. Add/release resources (receptors, beams) to/from the sub‐array. TM accesses directly the Mid.CSP Sub-array in order to add/release resources to/from sub‐arrays. The TM requests are validated by the Mid.CSP sub-array relying on the information maintained by the Mid.CSP Controller and Capability.

  2. Set the frequency Band.

  3. Configure the sub‐array Processing Modes using command Configure(). This includes configuration of the Capabilities used by the sub‐array.

  4. Start/stop scan execution.

  5. Set ‘engineering’ parameters for the sub‐arrays and Capabilities.

Receptors assignment

The assignment of receptors to a MID sub-array is performed in advance of a scan configuration.

Up to 197 receptors can be assigned to one sub-array.

Receptors assignment to a sub-array is exclusive: one receptor can be assigned only to one sub-array.

During assignment, the Mid.CSP sub-array performs both formal and more deep checks.

Formal control: - The assigned receptor list is empty. In this case, a warning is logged. - The receptor names are compliant to ADR-32. In this case, a warning is logged for each receptor not compliant and the receptor is excluded from the assigned list. - The assigned receptor list contains duplicated receptors. In this case, the duplicated receptor is removed from the list and a warning is logged.

Deep control: - The requested receptors are in the list of the deployed ones. - the requested receptors are already associated to another sub-array. In this case a warning message is logged.

In order for a Mid.CSP sub-array to accept a scan configuration, at least one receptor must be assigned to the sub-array.

Tied-array beams

Depending on the selected processing mode, Search Beams, Timing Beams and Vlbi Beams must be assigned to a MID sub-array in advance of a scan.

TM may specify the number of beams to be used or, alternatively, identify the Capability instances to be used via their TANGO Fully Qualified Domain Name (FQDN).

Scan configuration

TM provides a complete scan configuration to a sub-array via an ASCII JSON encoded string.

When a complete and coherent scan configuration is received and the sub-array configuration (or re-configuration) completed, the sub-array it’s ready to observe.

Once configured, Mid.CSP keeps the sub‐array and scan configuration until one of the following occurs:

  • Mid.CSP receives a command to release resources (receptors, FSPs, Search Beams and/or Timing Beams) used by the sub‐array.

  • Mid.CSP receives a command to transition to low‐power state (state=STANDBY); in which case the active scans end and resources are released.

  • Mid.CSP receives a command to shut‐down (OFF)

  • Mid.CSP monitor and control function fails so that the configuration is lost.

Inherent Capabilities

The following inherent Capabilities that correspond to Processing Modes, and are configured via scan configuration:

  1. Correlation

  2. PSS

  3. PST

  4. VLBI (beamforming and correlation in support of VLBI)

  5. Transient Data Capture.

When a sub‐array does not perform correlation, its Capability Correlation is OFF; the same applies for all Processing Modes.

Control and Monitoring

Each Mid.CSP Sub-array maintains and report the status and state transitions for the Mid.CSP sub-array as a whole and for the individual assigned resources.

In addition to pre-configured status reporting, a Mid.CSP Sub-array makes provision for the TM and any authorized client, to obtain the value of any sub-array attribute.

Mid.CSP Sub-array TANGO Device name

The Mid.CSP Sub-array TANGO Device name is defined in the document “SKA1 TANGO Naming Conventions”:

mid-csp/subarray/XY

where XY is a two digit number in range [01,..,16].

Mid.CSP Sub-array operational state

Mid.CSP Sub-array intelligently rolls‐up the operational state of all components used by the sub‐array and reports the overall operational state for the sub‐array.

The Mid.CSP Sub-array supports the following sub-set of the TANGO Device states:

  • UNKNOWN: Mid.CSP sub-array is unresponsive, e.g. due to communication loss.

  • OFF: The sub-array is not enabled to perform signal processing. The sub‐array is ‘empty’.

  • INIT: Initialization of the monitor and control network,equipment and functionality is being performed.

  • DISABLE: Mid.CSP sub-array is administratively disabled, basic monitor and control functionality is available but signal processing functionality is not available.

  • ON: The sub-array is enabled to perform signal processing. The sub‐array observing state is EMPTY if receptors have not been assigned to the sub‐array, yet.

  • ALARM: Quality Factor for at least one attribute crossed the ALARM threshold. Part of functionality may be unavailable.

  • FAULT: Unrecoverable fault has been detected. The sub‐array is not available for use and maintainer/operator intervention might be required.

Mid.CSP Sub-array observing state

The sub‐array Observing State indicates status related to scan configuration and execution.

The Mid.CSP Sub-array observing state adheres to the State Machine defined by ADR-8.

Mid.CSP Sub-array TANGO Device Commands

Mid.CSP Sub-array implements the standard set of commands as specified in:

  • Standard set of TANGO Device commands as defined in TANGO User Manual

  • Standard set of SKA TANGO Device commands

  • Command specific to Mid.CSP Sub-array as described in API section.

Mid.CSP makes provision for TM to request state transitions for individual sub‐systems and/or Capabilities.

Mid.CSP Sub-array TANGO Device Attributes

Mid.CSP Sub-array implements the standard set of attributes as specified in:

  • Standard set of TANGO Device attributes as defined in TANGO User Manual

  • The standard set of SKA TANGO Device attributes as defined for the SKA Sub-array TANGO Device.

  • Attributes to Mid.CSP Sub-array as described in API section.

Virtually all parameters provided in scan configuration are exposed as attributes either by Mid.CSP Sub-array or by individual Capabilities.

Mid.CSP LMC AlarmHandler

One of the CSP LMC role is to provide the capability to manage the abnormal situations caused by the CSP subsystem hardware and software components when the SKA telescope is operating. The CSP subsystem is responsible to evaluate the monitored attributes and the conditions generated by all its TANGO Devices (including sub-systems) and to define appropriate alerts for the notifications to report to TM.

The handling of these situations is managed by a specific TANGO device: the SKA Alarm Handler. This device is based on Elettra Alarm Handler provided by the TANGO community and the related document is available at <https://gitlab.com/ska-telescope/ska-tango-alarmhandler>,

Any abnormal condition individuated by the CSP Alarm Handler is defined via an alarm rule that is a logical combination of attribute values, quality factors, device states and modes across one or more CSP devices, as well as of the Element Alerts implemented by the CSP sub-systems, in case these implement their own alarm system. The error conditions generated by the CSP.LMC devices as well those raised by the CSP subsystems, are evaluted and alarms are triggered on the basis of pre-defined aggregation rules. An alarm attribute is implemented in the ALarmHandler devices for all those alarms that have to be reported to the Central Alarm Handler for further examination.

Mid.CSP AlarmHandler deployment

The Mid CSP subsystem implements its own Alarm Handler that is deployed along with the other devices, in the Mid CSP namespace, via a separate HELM chart (ska-csp-mid-alarm). This chart has to be added to the umbrella chart of the subsystem, as shown below

- name: ska-csp-lmc-mid
  version: 0.17.2
  repository: "file://../ska-csp-lmc-mid"
- name: ska-csp-mid-alarm
  version: 0.1.0
  condition: ska-csp-mid-alarm.enabled

Mid.CSP AlarmHandler TANGO Device name

The Mid.CSP AlarmHandler TANGO Device name is registered into the TANGO DB with the following name:

mid-csp/alarm/0

Mid.CSP Alarm Handler configuration

The Alarm Handler has to be configured by loading the alarms rules into the devices. This is done using the Load command and specifying as input a string with the alarm rule.

For example:

alarm = Device('mid-csp/alarm/0')
alarm.load('tag=testcsp;formula=(mid-csp/control/0/state == FAULT);priority=log;group=none;message="Mid Csp Controller is in FAULT!")

Once successfully loaded, a new TANGO attribute named testcsp is exposed by the device.

The SKA Tango Alarm Handler project provides a UI that allows a user to load a file with all the alarm rules needed to configure the device.

Currently, the MID CSP.LMC Alarm Handler can be configured only manually, loading the alarm rules via the Load command. Automatic configuration is planned as future improvement.

Test of the Mid CSP.LMC Alarm Handler

As starting point, the Mid CSP.LMC Controller and Subarray devices expose a testAlarm R/W TANGO attribute (boolean flag). In the alarm rule file is defined the following formula:

tag=cspTestAlarm;formula=(mid-csp/control/0/testAlarm | mid-csp/subarray/01/testAlarm);priority=high;group=none;message="Test Alarm for CSP.LMC devices"

so that if the alarm flag is set, an alarm with high priority is raised by the Mid CSP.LMC AlarmHandler. The integration test described in the alarm.feature file tests this case.