Kubernetes

Low CBF Processor is designed to be able to run under Kubernetes (k8s).

We have two container variants: AMI & XRT, named for the interface used to communicate between FPGA and Processor software.

In each case, k8s resource allocation is used to ensure each Processor pod has exclusive access to an FPGA.

AMI

“AMI” is an abbreviation for AVED Management Interface. “AVED” is an abbreviation for AMD Alveo™ Versal™ Example Design.

This interface is used for the V80 FPGA hardware.

AMI docs: https://xilinx.github.io/AVED/

Container name: ska-low-cbf-proc-ami

k8s Device Plugin

We use generic-device-plugin.

A sample configuration file is in resources/k8s-generic-device-plugin/ami-fpga.yaml. This works by grouping the /dev/ami0 and /dev/ami1 devices and granting access to them. It can easily be extended to suit multiple cards.

Deploy the plugin by running:

kubectl apply -f resources/k8s-generic-device-plugin/ami-fpga.yaml

k8s resource name: squat.ai/ami-fpga

To request an AMI FPGA:

resources:
  limits:
    squat.ai/ami-fpga: 1

An example pod specification is in resources/k8s-generic-device-plugin/ami-k8s-test.yaml.

Important Note re. ami_tool

We aren’t sure if we are allowed to distribute the ami_tool executable, so it is not included in the source code repository or container images.

Note

ami_tool is required for the current FPGA interface to V80 FPGAs!

Acquire the relevant AVED “Deployment Archive”, e.g. the latest should be here.

Then copy the ami_tool executable into the container:

kubectl cp /usr/local/bin/ami_tool <namespace>/<pod>:/app/ami_tool

You may make ami_tool available to the container via some other means, e.g. a host mount.

XRT

“XRT” is an abbreviation for Xilinx Runtime

This interface is used for the U55C FPGA hardware.

Container name: ska-low-cbf-proc-xrt

k8s Device Plugin

Xilinx “FPGA as a Service”

Resource name: amd.com/xilinx_u55c_gen3x16_xdma_base_3-0 (the ‘3’ near the end is subject to change with FPGA platform version)

To request an XRT FPGA:

resources:
  limits:
    amd.com/xilinx_u55c_gen3x16_xdma_base_3-0: 1