LowCbfProcessor

LowCbfProcessor is a Tango device server for monitoring and control of registers in the Low.CBF signal processing FPGAs. It’s a lightweight wrapper around an AlveoCL “core” object, which communicates to the Alveo FPGA modules using PyOpenCL. Processors are shared by and perform correlation or beamforming for multiple subarrays simultaneously.

The processor device uses the ska-low-cbf-fpga python package to represent registers within the FPGA design as “fields” and groupings of fields (“peripherals”) it can interact with. The ska-low-cbf-fpga package is a general framework for interacting with registers in Low.CBF FPGA designs. Registers within a FPGA are read from the address map file associated with each FPGA design (fpgamap_NNNNNNNN.py). The processor device can expose FPGA register fields (or higher-level abstractions of fields) as Tango attributes.

The processor device repository contains code to control and monitor two specific FPGA design personalities

  • Low.CBF PST Beamformer FPGAs

  • Low.CBF correlator FPGAs

Processor communication with Allocator

Processor Tango devices subscribe to Allocator events that represent the desired state of the processor device. On receipt of an event, a processor device will attempt to conform itself to the desired state. It will translate the attribute parameters into appropriate register values for the FPGA design personality. There are two attributes to which a Processor subscribes:

  • “internal_alveo” conveys per-Alveo data

  • “internal_subarray” conveys per-subarray settings that are used globally by Alveos

The Allocator “internal_alveo” attribute conveys information about the internal register settings of every Alveo. The attribute data is a JSON-encoded dictionary, using Alveo serial numbers as the key. If an Alveo card’s serial number is not present then the Alveo is not currently in use by any subarray. Each Alveo uses only the data listed under its own ID. The data includes the FPGA personality the Alveo should run and an abbreviated description of its register settings.

The Allocator’s “internal_subarray” attribute conveys information about the currently operating subarrays. The attribute data is a JSON-encoded dictionary using Subarray ID [1-16] as key. If a subarray’s key is not present the the subarray is not in use. Values for each subarray provide information about subarray stations, beams, frequencies, and destinations for the Alveo’s output products. The information is potentially used by, and common to, every Alveo.

Delay polynomials

When configured as a correlator, the Processor receives the Tango FQDN of the delay polynomial source for every subarray-beam it is processing via the “internal_subarray” attribute described previously. It subscribes to Tango change events immediately. Delay polynomials are received as they are produced by the Tango device (usually every 10 seconds) and queued for use at an appropriate time. The correlator monitors the packet counters of the incoming SPS data to determine the current J2000 time according to the data stream. Every few seconds it checks if it is time for the the next delay polynomial to be programmed into the FPGA, writing it into the FPGA and removing it from the queue. Because the processor only queue 12 delay polynomials, the delay polynomial source must not generate polynomials more than 12 validity intervals (currently 120 seconds, but see ADR-88) into the future. Likewise, because the processor should have a polynomial available when required, the source should generate polynomials in time to arrive at the correlator FPGA slightly before they are required. The “stats_delay” attribute provides information about delay polynomials in the FPGA registers and whether the polynomials are valid (ie not being used before start-of-validity and not being used after end-of-validity).

Design

Processors load a FPGA executable for the particular Personality (correlator, beamformer, etc) they are to run. The executable is downloaded from SKA CAR or Gitlab. Since the download takes longer time than a Tango command is allowed to execute, firmware download and load is handled by spawning a separate thread for the task, allowing the Tango thread to return. Because FPGA personality can change at any time, code is designed to receive requests to download a new personality any time, even while a download is in progress. This requires careful interlocking of request data between Tango and the download thread.

Register values in an Alveo are only updated when any subarray that the Processor is handling changes between scanning and not-scanning states and when delay polynomials reach end of validity, requiring new delay values. Scanning state is derived from “internal_subarray” events, and any event that indicates scanning state has changed cause registers to be reprogrammed immediately.

Data from “internal_alveo” events however is simply recorded for use later, i.e. when scanning state changes. The SKA observing state machine ensures that a subarray receives configuration (affecting “internal_alveo” data) before it begins to scan, so the last recorded version of “internal_alveo” event data always contains the required Alveo register configuration to be applied when an “internal_subarray” event is received.

Simulation Mode

Reading the SimulationMode attribute shows whether the currently-loaded personality is using a driver that

  • communicates with and affects FPGA operation (SimulationMode=False), or

  • only simulates the operation of a FPGA (SimulationMode=True)

Simulation mode can be written with a new value, but the value only takes effect when a new FPGA personality is loaded. Until that time, reads of the attribute will continue to indicate the simulation state of the currently-loaded personality.

The Processor device starts up with simulationMode = True, simulating a minimal FPGA register map.

Continuous Integration

The current CI tests of this device use DeviceTestContext and do not require a full Tango system.