CNIC Control Software

Documentation

Documentation Status

The documentation for this project can be found in the docs folder, or browsed in the SKA development portal:

Project Avatar (Repository Icon)

Scene icons created by Freepik - Flaticon

Installation

End User

Releases are published via the SKA Central Artefact Repository.

pip install ska-low-cbf-sw-cnic \
  --extra-index-url https://artefact.skao.int/repository/pypi-internal/simple

More up-to-date development packages are available in this project’s package registry. Note that a GitLab API token will be required for access.

Developer

This repository uses Poetry, a python package and dependency manager, and pre-commit, a framework for managing git-commit hooks used here to check that committed code follows the project style guide.

  1. Install Poetry. Be sure to use “install_poetry.py”, not “get_poetry.py” (the superseded installer).

  2. Run poetry install to install the project & its dependencies. Poetry uses the pyproject.toml file which lists required python tools and versions.

  3. Install pre-commit, which is a framework for managing actions invoked when git commit is run.

  4. Run pre-commit install to ensure your code will be correctly formatted when committing changes. Pre-commit sets up actions based on the .pre-commit-config.yaml file in the repository.

  5. A git hook is provided that may help comply with SKA commit message rules. You can install the hook with cp -s "$(pwd)/resources/git-hooks"/* .git/hooks. Once installed, the hook will insert commit messages to match the JIRA ticket from the branch name. e.g. On branch perentie-1350-new-base-classes:

ska-low-cbf$ git commit -m "Add git hook note to README"
Branch perentie-1350-new-base-classes
Inserting PERENTIE-1350 prefix
[perentie-1350-new-base-classes 3886657] PERENTIE-1350 Add git hook note to README
 1 file changed, 7 insertions(+)

You can see the modified message above, and confirming via the git log:

ska-low-cbf$ git log -n 1 --oneline
3886657 (HEAD -> perentie-1350-new-base-classes) PERENTIE-1350 Add git hook note to README

Repository Layout

Source code files are divided into subdirectories:

  • src/ska_low_cbf_sw_cnic - module root

    • icl - Instrument Control (Abstraction) Layer, generally one file per FPGA peripheral

    • util - Utilities, generally with command-line interfaces

Usage

You’ll need to source the XRT setup script before using these utilities.

source /opt/xilinx/xrt/setup.sh

The following utilities are provided, each can be launched directly from the command line.

Use the --help flag to get more detailed (and up-to-date) usage instructions.

Debug Console & Register Viewer

Executing the module directly provides a command line interface with debug console and interactive register view functions.

usage: cnic [-h] [-f KERNELS [KERNELS ...]] [-d DEVICES [DEVICES ...]] [-m MEMORIES [MEMORIES ...]] [-s SIMULATE [SIMULATE ...]]
            [-r REGISTERS] [-i] [-c] [-e EXEC] [--monitor] [--no-firmware-check] [--ptp-domain PTP_DOMAIN] [--ptp-source-b]
            [command [command ...]]

ska-low-cbf FPGA Command-Line Utility

positional arguments:
  command
                        EXPERIMENTAL COMMAND INTERFACE
                        Available commands:
                        	monitor
                        	tx <filename> [rate [start_time [stop_time]]]
                        	rx <filename> <packet size> <n packets>

optional arguments:
  -h, --help            show this help message and exit
  -f KERNELS [KERNELS ...], --kernel KERNELS [KERNELS ...]
                        path to xclbin kernel file(s)
  -d DEVICES [DEVICES ...], --device DEVICES [DEVICES ...]
                        Bus:Device.Function PCIe addresses of card(s) to use
  -m MEMORIES [MEMORIES ...], --memory MEMORIES [MEMORIES ...]
                        HBM configuration(s) per card - <size><unit><s|i>.
                                             size: int
                                             unit: k, M, G (powers of 1024)
                                             s: shared
                                             i: FPGA internal
                                             e.g. '128Ms:1Gi'
  -s SIMULATE [SIMULATE ...], --simulate SIMULATE [SIMULATE ...]
                        path(s) to fpgamap_nnnnnnnn.py file(s) to simulate
  -r REGISTERS, --registers REGISTERS
                        register setting text file to load
  -i, --interactive     use interactive interface (unsupported!)
  -c, --console         use IPython console
  -e EXEC, --exec EXEC  Python file to execute

CNIC:
  CNIC-specific Arguments

  --monitor, -o         Launch monitoring interface
  --no-firmware-check   Skip firmware personality & version check
  --ptp-domain PTP_DOMAIN
                        PTP domain. Default: 24
  --ptp-source-b        Use PTP B? (Default: No)

Inside the debug console, use fpgas or fpga to access the FPGA(s).

Transmit a PCAP(NG) File

An experimental (read: subject to change!) command-line interface is available. Use cnic <kernel and memory flags> tx <filename>.

e.g.

cnic -d 3 -m "4095Ms:4095Ms:4095Ms:4095Ms" -f cnic.xclbin -- tx my_pcap_file.pcap

The program will return immediately, without waiting for the FPGA to send the packets.

Receive a PCAP(NG) File

An experimental (read: subject to change!) command-line interface is available. Use cnic <kernel and memory flags> rx <filename> <packet size> <n packets>.

e.g.

cnic -d 3 -m "4095Ms:4095Ms:4095Ms:4095Ms" -f cnic.xclbin -- rx new_pcap_file.pcap 2048 100

The program will block until the expected number of packets is received or the HBM memory is full.

The number of packets setting is a software trigger, when the trigger is hit all packets in HBM will be dumped. Your PCAP file may contain more packets than the trigger value.

Duplex Mode

As of CNIC firmware 0.1.5, full duplex operation is available. Enable duplex mode in the HBM Packet Controller peripheral prior to configuring transmit or receive.

Example for debug console:

fpga.hbm_pktcontroller.duplex = True
fpga.receive_pcap("duplex_capture.pcap", 2112, 20)
fpga.transmit_pcap("tests/codif_sample.pcapng")

In duplex mode, the packet data to transmit is loaded into the 3rd HBM buffer. Make sure to use “shared” mode for at least buffers 1 & 3!

Monitor Status

An experimental (read: subject to change!) command-line interface is available. Use cnic <kernel and memory flags> monitor.

e.g.

cnic -d 3 -m "4095Ms:4095Ms:4095Ms:4095Ms" -f cnic.xclbin monitor

The monitor interface is a full-screen terminal application. Use Ctrl-C to exit.

change_port

Change the port number in a pcap file.

Warning the input file is loaded into memory in full.

change_port [-h] [--port PORT] [--output OUTPUT] input

compare_pcap

Find differences between two pcap files, checking only packets with a specified destination port. Exits with non-zero status if differences found.

compare_pcap [-h] [--packets PACKETS] [--dport DPORT] [--report REPORT] input input

eth_interface_rate

Monitor the transmit & receive rates on an ethernet interface.

Warnings

  • uses the sudo command

  • activates promiscuous mode on the interface

  • requires ethtool

eth_interface_rate [-h] interface

Road Map

Some ideas for future work… consider all details suggestions subject to change, they’re not set in stone!

  • Controlling multiple FPGA cards with one instance of the program would be viable. Could either use a loop over a list of CnicFpga objects or invent some FPGA group object to handle it.

  • There really should be a lot more tests!

    • We might need ArgsSimulator to simulate HBM in order to test without an FPGA

    • Adding tests that exercise a real FPGA may be easier, and a good idea in any case…

  • Configuration of inter-packet gaps (i.e. jitter, maybe a Poisson distribution)

  • Record statistics over time, so they can be graphed

    • Sampling statistics in the FPGA would be more accurate, we could download when ready, in batches, whatever