alveo2gemini¶
This directory contains software that bridges the OpenCL PCI interface of Xilinx Alveo cards to the Gemini Network Protocol. It allows Alveo cards to be controlled in the same way as Gemini FPGA cards are controlled. The software is responsible for loading FPGA firmware into the card and then executing an openCL kernel, implemented in FPGA RTL, to read and write registers in the RTL design.
hbm2file¶
The directory contains source code for the hbm2file application which works with alveo2gemini to dump Alveo card HBM contents.
Dependencies¶
Install openCL sudo apt-get install opencl-headers
Build¶
To build the software, Xilinx Vitis software suite must first be installed and the following environment variables need to be defined to provide the path to Xilinx software
XILINX_VITIS (eg export XILINX_VITIS=/tools/Xilinx/Vitis/2020.1)
XILINX_VIVADO (eg export XILINX_VIVADO=/tools/Xilinx/Vivado/2020.1) need to setup XRT reference as well source /opt/xilinx/xrt/setup.sh
The software can be built by issuing the ‘make’ command
Usage¶
The alveo2gemini executable has several optional parameters:
-f fpgaXclbinFilename (xclbin file containing FPGA firmware to be run)
-k kernelName (name of the OpenCL kernel within the FPGA firmware)
-p serverPort (port number that Gemini Server should listen on)
-b argsBaseAddr (address of Vitis shared memory in ARGS address space)
-w (wait after launch for FPGA debugger to be attached)
-v (show build date and git version, then exit)
If no parameters are specified, the defaults are:
-f aug28.xclbin -k vitisAccelCore -p 30333 -b 0x8000
The hbm2file executable has three optional command line parameters:
-f outputFilename (name of file to which HBM contents are dumped)
-i hbm_index (zero-based index indicating which HBM to dump)
-v (show build date and git version, then exit)
If no parameters are specified, the defaults are:
-f hbm.dat -i 0
Internal Details¶
Main routine (file: main.cpp)¶
The main routine is the top-level of the alveo2gemini executable. The main loop that handles the gemini packets received from clients runs until a user kills it or an unexpected error occurs. Its pseudo-code outline is:
Create an “Alveo_cl” object which initialises OpenCL, loading FPGA with its bitfile
Create 1st UDP socket that will broadcast Gemini “event” packets to all-network-hosts, port 30001
Construct a Gemini “publish” event payload that can be sent every 2 seconds by the socket
Create 2nd UDP socket, bound to port 30333 by default, that Gemini clients will connect to
While (user hasn’t sent ctrl-c to kill the program) do:
wait up to 50msec for a packet to be received from a gemini client on the 2nd socket
if a packet was received:
pass packet payload to the gemini-protocol-handler
send the gemini-protocol-handler response back to the user via the 2nd socket
if more than 2 seconds have elapsed since last publish packet was sent
send a publish packet via the 1st socket
Gemini-protocol-handler (files: gemini.h, gemini.cpp, c++ class: Gemini)¶
The gemini-protocol-handler (gemini.cpp) processes gemini client packets in a way that mimics the processing done by gemini FPGA cards. The one key difference is that the FPGA implementation of the processing includes packet-retry mechanisms to deal with lost network packets. This handler does not retry and drops retries because there can be no lost packets over the PCI bus connection to the FPGA card.
The gemini-protocol-handler has a single “packet_in” function. The function calls the OpenCl object’s “reader” and “writer” methods to read/write to gemini registers in the FPGA. In pseudo-code, the operations that the packet_in method performs are:
check that packet payload is actually gemini protocol, and drop if not
if the packet is a Gemini “Connect” operation
create an “ACK” payload that’s returned to the caller
If the packet is a Gemini “Read Inc” operation
invoke OpenCL to perform a register read operation on the Alveo card
put the read results into a Gemini packet that’s returned to the caller
If the packet is a Gemini “Write Inc” operation
invoke OpenCL to perform a register write operation on the Alveo card
create an “ACK” payload that’s returned to the caller
If the packet is any other kind of Gemini packet (unknown type)
drop the packet, no response to the caller
Alveo OpenCL interface (file: alveo_cl.h, alveo_cl.cpp, c++ class: Alveo_cl)¶
This class contains methods that call OpenCL to communicate with an Alveo card. It has three key methods:
Initialisation:
loads FPGA bitfile,
creates gigabyte buffers in HBM in the FPGA and on the host
creates Posix Shared memory used for dumping HBM contents for debug
write:
performs OpenCL PCI-bus transactions that write registers in the FPGA
read:
performs OpenCL PCI-bus transactions that read registers from the FPGA
also handles dump requests for any of the HBM buffers
performs OpenCL PCI bus transactions that copy HBM contents to host
copies buffer data to shared memory, accessible by other applications