Talon DX Script

class talondx.PowerSwitch[source]

Class to manage the network-controlled power switch.

off()[source]

Check the switch state, then powers it off if the switch state is not already off; otherwise no action. NOTE: it is strongly recommended to shut down the Talon boards prior to powering off.

state()[source]

Queries the power switch state and returns the result as PowerSwitchState enum

class talondx.PowerSwitchState(value)[source]

An enumeration.

class talondx.Target(value)[source]

An enumeration.

class talondx.Version(filename)[source]

Class to facilitate extracting and comparing version numbers in filenames.

Parameters:

filename – string containing a version substring in the x.y.z format, where x,y,z are numbers.

match(ver)[source]

Compare two Version object and return true if the versions match.

Parameters:

ver – Version object being compared to this one.

talondx.get_device_fqdn_list()[source]

Get full list of fully-qualified device names (FQDNs) from the Tango database, excluding “sys” and “dserver” names.

Ref: https://tango-controls.readthedocs.io/en/latest/tutorials-and-howtos/how-tos/how-to-pytango.html

Returns:

alphabetically-sorted list of FQDNs (str)

talondx.get_device_fqdns(devices, server_inst)[source]

Generate list of fully-qualified device names (FQDNs) from Tango database for the given list of devices and server instance.

Ref: https://tango-controls.readthedocs.io/en/latest/tutorials-and-howtos/how-tos/how-to-pytango.html

Parameters:

  • devices (list of str) – device names

  • server_inst (string) – server instance

Returns:

list of FQDNs (str)

talondx.get_device_status(config_commands)[source]

Reads and displays the state and status of each HPS Tango device running on the Talon DX boards, as specified in the configuration commands – ref “config_commands” in the talondx-config JSON file.

Parameters:

config_commands (str) – JSON array of configure commands

talondx.get_device_version_info(config_commands)[source]

Reads and displays the dsVersionId, dsBuildDateTime, and dsGitCommitHash attributes of each HPS Tango device running on the Talon DX boards, as specified in the configuration commands – ref “config_commands” in the talondx-config JSON file.

Parameters:

config_commands (str) – JSON array of configure commands

Talon DX Config

TalonDxConfig Class

class talondx_config.talondx_config.TalonDxConfig(config_file)[source]

TalonDxConfig facilitates loading and validation of the Talon DX Configuration JSON file (see schema for details).

Parameters:

config_file (string) – filename of the JSON configuration file

config_commands()[source]

Extracts and returns the “config_commands” section of the configuration file that specifies the configuration commands that are sent from the MCS to the Talon DX HPS Master device.

ds_binaries()[source]

Extracts and returns the “ds_binaries” section of the configuration file that specifies the Tango DS binaries to be downloaded, and where to get them.

export_config(export_path)[source]

Exports the Talon DX Configuration JSON to a file with same name as that used to construct this object. Export will overwrite if the file already exists.

Parameters:

export_path (string) – destination path of exported configuration file.

fpga_bitstreams()[source]

Extracts and returns the “fpga_bitstreams” section of the configuration file that specifies which FPGA bitstreams to download, and where to get them.

tango_db()[source]

Extracts and returns the “tango_db” section of the configuration file that contains the device server specifications for populating the Tango DB.

Schema

{
    "type": "object",
    "properties": {
        "ds_binaries": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "source": {"type": "string", "enum": ["conan", "git"]},
                    "conan": {"$ref": "#/$defs/conan"},
                    "git": {"$ref": "#/$defs/git"}
                },
                "required": [
                    "name",
                    "source"
                ],
                "additionalProperties": false
            }
        },
        "fpga_bitstreams": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "source": {"type": "string", "enum": ["raw", "git"]},
                    "version": {"type": "string"},
                    "raw": {"$ref": "#/$defs/raw"},
                    "git": {"$ref": "#/$defs/git"}
                },
                "required": [
                    "source",
                    "version"
                ],
                "additionalProperties": false
            }
        },
        "config_commands": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "description": {"type": "string"},
                    "target": {"type": "string"},
                    "ip_address": {"type": "string"},
                    "talon_first_connect_timeout": {"type": "integer"},
                    "ds_hps_master_fqdn": {"type": "string"},
                    "fpga_path": {"type": "string"},
                    "fpga_dtb_name": {"type": "string"},
                    "fpga_rbf_name": {"type": "string"},
                    "fpga_label": {"type": "string"},
                    "ds_path": {"type": "string"},
                    "server_instance": {"type": "string"},
                    "talon_lru_fqdn": {"type": "string"},
                    "ds_rdma_rx_fqdn": {"type": "string"},
                    "devices": {
                        "type": "array",
                        "items": {"type": "string"}
                    }
                },
                "required": [
                    "description",
                    "target",
                    "ip_address",
                    "ds_hps_master_fqdn",
                    "fpga_path",
                    "fpga_dtb_name",
                    "fpga_rbf_name",
                    "fpga_label",
                    "ds_path",
                    "server_instance",
                    "devices"
                ],
                "additionalProperties": false
            }
        },
        "tango_db": {
            "type": "object",
            "properties": {
                "db_servers": {
                    "type": "array"
                }
            }
        }
    },
    "required": [
        "ds_binaries", 
        "fpga_bitstreams", 
        "config_commands", 
        "tango_db"
    ],
    "$defs": {
        "conan": {
            "type": "object",
            "properties": {
                "package_name": {"type": "string"},
                "user": {"type": "string"},
                "channel": {"type": "string"},
                "version": {"type": "string"},
                "profile": {
                    "type": "string", 
                    "enum": ["conan_aarch64_profile.txt", "conan_x86_profile.txt"]
                }
            },
            "required": [
                "package_name",
                "user",
                "channel",
                "version",
                "profile"
            ],
            "additionalProperties": false
        },
        "raw": {
            "type": "object",
            "properties": {
                "group": {"type": "string"},
                "base_filename": {"type": "string"}
            },
            "required": [
                "base_filename"
            ],
            "additionalProperties": false
        },
        "git": {
            "type": "object",
            "properties": {
                "git_project_id": {"type": "integer"},
                "git_branch": {"type": "string"},
                "git_pipeline_job": {"type": "string"}
            },
            "required": [
                "git_project_id",
                "git_branch",
                "git_pipeline_job"
            ],
            "additionalProperties": false
        }
    }
}

DB Populate

Conan

ConanWrapper Class

class conan_local.conan_wrapper.ConanWrapper(folder)[source]

ConanWrapper provides a Python interface to the shell commands required to download Conan packages. There is a Python API, but it is not documented, and according to: https://github.com/conan-io/conan/issues/6315 The python api is not documented nor stable. It might change at any time and break your scripts. This class can be updated to use the Python API if/when that makes sense.

Parameters:

folder (string) – destination path for downloaded (deployed) conan packages.

static clear_local_cache()[source]

Removes all packages and binaries from the local cache.

download_package(pkg_name, version, user, channel, profile, timeout=60)[source]

Run conan install to download the conan package and deploy it to the folder specified at construction.

Parameters:

  • pkg_name (string) – conan package name

  • version (string) – conan package version number

  • user (string) – user name of conan package

  • channel (string) – conan package channel name

  • profile (string) – name of text file containing the conan profile used to deploy the download package

  • timeout (int) – download timeout in seconds

static search_local_cache()[source]

Searches local cache for package recipes and binaries.

static version()[source]

Returns conan version.

Conan Profiles

Cross-compiled (HPS) Tango Devices

target_host=aarch64-linux-gnu
toolchain=/usr/$target_host
cc_compiler=gcc
cxx_compiler=g++

[settings]
os=Linux
arch=armv8
compiler=gcc
compiler.version=7
compiler.libcxx=libstdc++11
build_type=Release

[env]
CONAN_CMAKE_FIND_ROOT_PATH=$toolchain
CONAN_CMAKE_SYSROOT=$toolchain
PATH=[$toolchain/bin]
CHOST=$target_host
AR=$target_host-ar
AS=$target_host-as
RANLIB=$target_host-ranlib
LD=$target_host-ld
STRIP=$target_host-strip
CC=$target_host-$cc_compiler
CXX=$target_host-$cxx_compiler
CXXFLAGS=-I"$toolchain/include"

Native-compiled (Linux server) Tango Devices

target_host=x86_64
toolchain=/usr/bin
standalone_toolchain=/usr
cc_compiler=gcc
cxx_compiler=g++

[settings]
os=Linux
arch=x86_64
compiler=gcc
compiler.version=9
compiler.libcxx=libstdc++11
build_type=Release

[env]
PATH=[$standalone_toolchain/bin]
CHOST=$target_host
AR=ar
AS=as
RANLIB=ranlib
LD=ld
STRIP=strip
CC=$cc_compiler
CXX=$cxx_compiler
CXXFLAGS=-I"$standalone_toolchain/include"

Conan Remotes

{
    "remotes": [
     {
       "name": "ska",
       "url": "https://artefact.skatelescope.org/repository/conan-internal/",
       "verify_ssl": true
     },
     {
      "name": "conan.io",
      "url": "https://center.conan.io/",
      "verify_ssl": true
     }
    ]
}

Talon DX Log Consumer

The Talon DX Log Consumer is a Tango device intended to run on the host machine that connects to the Talon-DX boards. This Tango device is set up as a default logging target for all the Tango device servers running on the HPS of each Talon-DX board. When the HPS device servers output logs via the Tango Logging Service, the logs get transmitted to this log consumer device where they get converted to the SKA logging format and outputted once again via the SKA logging framework. In this way logs from the Talon-DX boards can be aggregated in once place and eventually shipped to the Elastic framework in the same way as logs from the Mid CBF Monitor and Control Software (MCS).

Note that eventually this Tango device will be moved to the Mid CBF MCS, and more instances of the device may be created to provide enough bandwidth for all the HPS device servers.

Connecting from HPS DS to the Log Consumer

The Talon-DX boards connect to the host machine (currently known as the Dell Server) over a single Ethernet connection. The IP address of the Dell Server on this connection is 169.254.100.88 and all outgoing traffic from the Talon-DX boards must be addressed to this IP.

When the log consumer starts up on the Dell server, the OmniORB end point (IP address and port) it is assigned is local to the Dell server (i.e. IP address 142.73.34.173, arbitrary port). Since the Talon boards are unable to connect to this IP address. we need to manually publish a different endpoint when starting up the log consumer that is visible to the HPS devices.

The following ORB arguments are used (see the make target talondx-log-consumer):

  • -ORBendPointPublish giop:tcp:169.254.100.88:60721: Exposes this IP address and port to all clients of this Tango device. When the HPS device servers contact the database to get the network information of the log consumer, this is the IP address and port that is returned. The IP addresses matches that of the Ethernet connection to the Dell server, allowing the HPS device servers to direct their messages across that interface.

  • -ORBendPoint giop:tcp:142.73.34.173:60721: Assigns the IP address and port that the log consumer device is actually running on. This needs to be manually assigned since an iptables mapping rule was created on the Dell server to route any TCP traffic coming in on 169.254.100.88:60721 to 142.73.34.173:60721.

Some important notes:

  • Due to the end point publishing, no Tango devices running on the Dell server will be able to connect to the log consumer (including being able to configure the device from Jive). This is because the published IP address is not accessible on the Dell server. There may be a way to publish multiple endpoints, but this needs further investigation.

  • If the log consumer device cannot be started due to an OmniORB exception saying that the end point cannot be created, it is possible that the 142.73.34.173 needs to change to something else. It is not yet clear why this can happen. To change it do the following:

    • Remove the ORB arguments from the talondx-log-consumer make target, and then start the log consumer.

    • Open up Jive and look at what IP address is automatically assigned to the log consumer device. This is the IP address that we now need to use for the endpoint.

    • Find the iptables rule that maps 169.254.100.88:60721 to 142.73.34.173:60721, and change it to the new IP address.

    • Add the ORB arguments back in, using the correct IP address for the end point.

Automated Script

The automated script is a method to deploy the MCS system inside the minikube and execute commands through the engineering console without relying on make commands in a given git repository. All images and containers are pulled directly from CAR.

Preconditions

  • The script must be run as a user with passwordless root permission.

  • The script must be run from the …/automation directory.

  • Latest stable MCS and Engineering Console image versions are known hard-coded in script.

Key Files

The following files are necessary to run the automated script:

  • orchestration.sh: The entry point for a cronjob.

    • Ensures more than one instance of the script is not running.

    • Creates the test result directories.

  • setup.sh: Sets up the test environment.

    • Records the test configuration.

    • Starts Minikube.

    • Programs the talon boards.

  • script.sh.multiboard: The test. See “Current Outcomes” below.

Current Outcomes

Using a git and makefile detached script to automate the following:

  • MID CBF MCS Deployment

  • Use Engineering Console to:

    • Configure the Tango DB inside MCS

    • Turn on the Talon Boards using the LMC Interface

    • Check the Talon DS Versions to verify status

    • Generate BITE Data

    • Replay the BITE Data through the board back into the primary server

    • Capture the data on the Engineering Console

    • Configure the VCC Bands

    • Use the Serial Loop-back to Send the BITE Data back through the board into the VCC Firmware IP Blocks

    • Use the RDMA Tx to send the VCC Data to the RDMA Rx

  • Run the automated script nightly as a regression test.

Future Outcomes

The features of the automated script will be extended to the following:

  • Verify the data captured by the RDMA Rx

The current automated script is run nightly and saves the test results locally. The aim is to send the results to JIRA X-RAY, as well as to generate reports with captured data and plots.

Running the Script

Refer to the Automated Script Confluence Page