Getting started

Implementing a processor

To write a new processor you need to:

• Install this package: pip install --extra-index-url=https://artefact.skao.int/repository/pypi-internal/simple ska-sdp-realtime-receive-processors.

• Create a new class that derives from realtime.receive.processors.sdp.base_processor.BaseProcessor and implement the necessary methods (click to see the signature).

  • create()

  • process()

  • timeout()

  • close()

Writing files

When a processor writes a file, it should call storage() to get access to the underlying storage, and declare_new_file() to register the new file to create, keeping a reference to the returned File object. Once the file has been completed, its update_status() method should be called with the final status.

In-built processors

This package also comes with some in-built processors that can be utilised. See In-built Processors for available implementations.

Running a processor

This package provides not only the base class for all plasma processors, but also the program used to load and run them. This program is called plasma-processor.

plasma-processor loads a user-provided processor class name (its only required parameter) and sets up the necessary infrastructure to connect it to a Plasma store. To run your processor execute plasma-processor <your-fully-classified-class-name> [args]

To see all available options, go to the CLI interface documentation.

Docker

A docker image is generated in GitLab CI and stored in the SKAO CAR. The default docker image will run the MSWriterProcessor on startup with only the output file being required.

docker compose build

Can be used to build a docker image.

The following will start an interactive MSWriterProcessor

docker run -it --rm --mount type=bind,source=/full-path-to-plasma-socket,destination=/tmp/plasma <ms output>

An alternative processor can be run by changing the entrypoint

docker run -it --rm --mount type=bind,source=/full-path-to-plasma-socket,destination=/tmp/plasma --entrypoint plasma-processor <processor class> <args>