CSP.LMC Observation Supervisor

class ska_csp_lmc_common.observation.observation_supervisor.CspObservationSupervisor(*, model: SubarrayObsStateModel, policy: CommandAwareObservationPolicy, store: ComponentSnapshotStore, update_attribute: Callable[[str, Any], None], get_obs_modes: Callable[[], list], logger: Logger, debounce_s: float = 0.1, max_latency: float = 0.3, reconcile_timeout_s=0.5)

Bases: ObservationSupervisor

Supervisor implementation focused on _evaluate_and_publish.

The base ObservationSupervisor still provides debounce and max-latency timing plus the background tick thread.

Each evaluation cycle takes a subsystem snapshot, asks the model to compute the aggregated candidate state, applies the observation policy, and then either latches a consistency FAULT or applies the final observation state through the model.

The ObsStateModel remains the single authority for de-duplication, precedence, and publication of obsState changes.

on_subsystem_event(component: ObservingComponent, obs_state: ska_control_model.ObsState | None = None, **kwargs) → None

Entry point for subsystem ObsState events from EventManager.

It handles the ingestion, debouncing, and orchestration of the state evaluation.

Parameters:

  • component – The component that generated the event.

  • obs_state – The new ObsState value.

  • kwargs – Additional arguments.

on_model_update(obs_state: ska_control_model.ObsState) → None

Handles state change notifications originating from the ObsStateModel itself (e.g., entering a transitional state like CONFIGURING).

These updates are considered high-priority and will always trigger an immediate evaluation, bypassing the debounce delay.

clear_consistency_fault() → None

Clear consistency-related fault state and diagnostic attributes.

If the model is currently latched in FAULT due to a consistency violation, that fault condition is cleared. The supervisor also resets the internal consistency flag and message and publishes the cleared diagnostic attributes.

This method only clears faults caused by consistency evaluation. It does not clear faults caused by subsystem or internal errors.

_evaluate_and_publish(*, force: bool = False, reconcile_timeout: bool = False, refreshed_after_reconcile: bool = False) → EvaluationOutcome

Evaluate the CSP subarray observation state.

This method takes a snapshot of subsystem states, evaluates aggregation and consistency policies, manages hard faults, and applies the resulting ObsState to the ObsStateModel.

The evaluation follows these steps:

1. Take an atomic snapshot of all subsystem ObsState values from the StateStore and update the subarray aggregation model via evaluate_states(…). The aggregated candidate state is stored in the model but not published.

2. Apply the CSP scan consistency policy to the aggregated candidate state, the active observation modes, and the full subsystem snapshot to detect hard inconsistencies or invalid combinations.

3. Manage FAULT transitions using a single fault mechanism (component_fault):

  • If the policy reports a hard fault, the subarray is forced into

    FAULT with cause CONSISTENCY. This state is latched until an explicit recovery action.

  • If no hard fault is reported and the subarray is currently in

    FAULT due to a consistency violation, the FAULT state is cleared automatically.

Fault-related diagnostic information (consistency flag and error message) is updated as part of this step.

4. Apply the final subarray ObsState to the ObsStateModel via apply_final_state(…). The model is responsible for precedence, de-duplication, and publication to the TANGO attribute if the value changes.

This method does not clear FAULT conditions caused by subsystem or internal errors; those require an explicit recovery action (e.g. Restart).

Parameters:

  • force – Whether the evaluation was triggered as a forced supervisor cycle rather than a normal debounced update. Present for compatibility with the generic supervisor interface.

  • reconcile_timeout – Whether the reconciliation timeout has expired for the current evaluation cycle.

  • refreshed_after_reconcile – If true, the current evaluation is being performed after a direct refresh requested at the end of a reconciliation window.

Returns:

The supervisor evaluation outcome indicating whether the state was applied, a wait is still required, or a fault was raised.

set_command_context(context: CommandContext) → None

Register a command context and make it the active one.

The supervisor delegates command lifecycle tracking to the internal command-context registry. The provided context becomes the active command context used during subsequent policy evaluations.

Parameters:

context – The command context associated with the command being started.

update_command_outcome(command_id: str, command_outcome: CommandOutcome, completion_time: datetime) → None

Update the outcome of a specific command context.

The matching command context is updated in the registry with the final outcome and completion time. If the context is successfully updated, an immediate observation re-evaluation is triggered.

Parameters:

  • command_id – Unique identifier of the command whose outcome is being reported.

  • command_outcome – Final aggregated outcome produced for the command.

  • completion_time – Timestamp at which the command completed.

get_scan_lifecycle_context() → CommandContext | None

Return the currently active Scan lifecycle context, if any.

A Scan lifecycle remains active while a Scan command is in progress and also after command completion when a retained Scan context still governs scheduled-scan behaviour. This includes the pending phase of a scheduled Scan where the CSP subarray may still report READY.

has_active_scan_lifecycle() → bool

Return whether a Scan lifecycle is currently active.

_clear_completed_command_context(retain_scan_context: bool = True) → None

Clear a completed active command context.

Scan contexts are retained only on normal policy paths. If the policy has already decided FAULT, the Scan lifecycle has ended abnormally and must not be retained.

Parameters:

retain_scan_context – whether the scan context has to be retained

_retain_or_clear_scan_context(context: CommandContext) → None

Retain or clear the retained Scan context after command completion.

This method is called when a completed active command context is being cleared by the observation supervisor.

For Scan commands, the context is passed to the command registry so it can be retained if it represents a completed Scan task. The registry is responsible for deciding whether the context is retainable, for example by checking that the command outcome status is TaskStatus.COMPLETED.

For commands that explicitly terminate or interrupt a Scan lifecycle, such as EndScan and Abort, the retained Scan context is cleared only when the command completed successfully with ResultCode.OK. If those commands complete with a non-OK result code, the retained context is preserved because the Scan lifecycle may not have been closed cleanly.

Parameters:

context – Completed command context being removed from the active command registry. If it represents a Scan command, it may be retained for post-command Scan lifecycle evaluation. If it represents a successful EndScan or Abort command, it may clear an existing retained Scan context.

_maybe_clear_legacy_retained_scan_context(*, final_state: ska_control_model.ObsState, previous_state: ska_control_model.ObsState, current_command_context: CommandContext | None, scan_lifecycle_context: CommandContext | None) → None

Clear retained Scan context once the relevant lifecycle is over.

Legacy Scan contexts have no schedule-based expiry, so they must be cleared explicitly once the subarray settles outside SCANNING.

Scheduled Scan contexts must remain retained across the initial pending phase where the CSP subarray can still report READY even though a scheduled Scan lifecycle is active. They are cleared only after the system has actually been in SCANNING and then settles back outside SCANNING, or through explicit successful EndScan/Abort handling.

The context is preserved while: - there is no retained scan context; - the retained scan context is scheduled; - the subarray is still SCANNING; - an active Scan command is still being reconciled.