Test Parameters

Summary

This folder contains prototype JSON definitions, JSON schemas, and code snippets to help understand parameterized testing of Mid CBF AA0.5 Use Cases. The test parameters are focused on the MID CBF Signal Chain Verification use case and corresponding test XTP-25202.

See tests.json for example test parameters. Each test is defined by a set of test parameters, including input test data, delay model configuration, subarray configuration, scan configuration, and test checkpoints.

To generate the test_parameters.json file and/or generate the signal_chain_verification.feature file for the MID CBF Signal Chain Verification Test, follow these steps:

Create a Python virtual environment and install the dependencies (if you already have a virtual environment created, skip the first three lines):

cd ~ # To navigate to your home directory on the dev server
pip install virtualenv
python3 -m virtualenv venv
source ~/venv/bin/activate
pip install -r requirements.txt

Clone this repo and run cd <path/to/ska-mid-cbf-system-tests/test_parameters>
Run ./test_parameters.py with at least one of the following flags: -t to generate the test_parameters.json file, and/or -f to generate the signal_chain_verification.feature file.

Mapping

System Tests assumes each talon target maps to a corresponding VCC and FSP, target 001 maps to VCC 001 and FSP 01.

CBF Input Data

See the cbf_input_data.json file inside the cbf_input_data folder for examples.

The CBF Input Data provides information for generating the Long Sequence Test Vector (LSTV), which is then fed into the signal chain. The cbf_input_data.json (link) specifies how many simulated receptors will be generating data and what that data will look like (based on bite configuration, k value, etc.)

Each configuration has a list of receptors to simulate. Each receptor must have the following parameters:

talon: A string containing the talon target number on which the data will be generated (eg. “003”).
dish_id: the dish id string (eg. “SKA063”)
sample_rate_k: the k value that modifies the BITE sample rate according to this equation:
- BITE Sample Rate = 3960000000 + k * 1800
- where 1800 is the frequency offset delta f.
- Summary: k value must be in range [1, 2222] and for TDC and an integer seconds LSTV must satisfy formula k = 56x + 16 where x is an integer
- BITE is limited to specific k values due to the consequences it has on the number of samples generated for an LSTV
  - 'Number of Samples' = 'LSTV Length in seconds' * 'BITE Sample Rate'
  - Limitations are on the corresponding number of samples as these must satisfy a number of requirements:
    - 'Number of Samples' must be a multiple of 18 to wrap nicely into the VCC (see CIP-4441 for additional context).
    - 'Number of Samples' must be a multiple of 24 to wrap nicely out of DDR4 (see CIP-4441 for additional context).
    - 'Number of Samples' must be a multiple of 192 so that the LSTV reads out the correct number of samples, with bursts of 8 (see CIP-4441 for additional context).
    - 'Number of Samples' must be a multiple of 21 such that the LSTV puts PPS at the start of packets correctly (see CIP-4441 for additional context).
  - For an integer length of LSTV seconds, the formula k = 56x + 16 such that x is an integer, satisfies this system of equations.
    - e.g. k can be set as: 16, 72, 128, …, 2088, 2144, 2200
    - Note: this was found by calculating all available k values for 1 second worth of LSTV samples, and identifying that multiplication of the samples by an integer does not effect congruency in modular arithmetic.
  - To calculate available k_values for a given lstv_length use the following script, replacing the value of lstv_length:
```
lstv_length = 1
for k in range(1, 2222):
	samples = lstv_length*(3.96e9 + 1800*k)
	if (samples % 18 == 0) and (samples % 24 == 0) and (samples % 192 == 0) and (samples % 21 == 0):
		print(k)
```
bite_config_id: name of the bite configuration to use from the bite_configs.json
delay_source_files (optional): a string containing the path to a folder containing a set of CSV files that define the delays that will be applied to the LSTV.
- If this field is not defined, no delays are applied to the LSTV. EC BITE will replay the LSTV with the initial timestamp of the data set to the current time.
- If this field is defined, the delays specified are applied to the LSTV. EC BITE will replay the LSTV with the initial timestamp of the data set to the following number of seconds since the SKA epoch (1999-12-31T23:59:28Z UTC): A) base_time from the dm_base_time.json file located two folders above delay_source_files + B) the first t_start in the CSV files.

Flowdown from Test IDs to Bite Configuration:

Each Test ID in tests.json is assigned to one particular set of CBF Input Data defined in cbf_input_data.json.
Each of these sets of CBF Input Data (eg. “talon-003 basic gaussian noise”) is assigned a BITE Configuration ID for each of its receptors, which corresponds to one of the BITE Configuration IDs defined in bite_configs.json.
Each of these configurations (i.e. “basic gaussian noise”, “ramp filtered gaussian noise and tone”, etc) contains a set of parameters to be used in configuring the BITE, which the BITE Client will write to the relevant device servers via Tango. In a given bite configuration, both polarizations of each gaussian noise source are assigned one of the filter types defined in filters.json, whose exact parameters will be parsed by the BITE Client.

BITE Configuration

See examples in bite_configs.json (link) within the cbf_input_data/bite_config_parameters folder.

Each BITE configuration in bite_configs.json has:

a description
an array of sources (of Gaussian noise)
an array of tone_gens

The sources array can be 1 element long (this documentation originally suggested 4 elements was possible, but this needs to be confirmed if it is desired to be used). Each element in the array must have the following parameters:

description: description of the source
pol_coupling_rho: the polarization coupler correlation coefficient in the range of [-1.0, 1.0]
- This rho value will be scaled by a factor of 2^16 and written directly to the polarization_coupler as the linear scaling coefficient of the cross-connected path (“alpha”); and it will be used in calculating the linear scaling coefficient sqrt(1-rho**2) of the straight-through path (“beta”) which will also be scaled by a factor of 2^16 and written to the polarization_coupler.
pol_Y_1_sample_delay: boolean true/false to respectively enable/disable the delay element in the cross-connected path of the polarization coupler
gaussian: object with two polarization properties (“pol_x” and “pol_y”) that are each given:
- seed: a randomization seed for Gaussian noise generation, which must be an integer in the range of 0 to 65445
- filter: string name of a filter defined in filters.json (link) that will be applied to that noise component
- noise_mean and noise_std (optional): the normalized mean and standard deviation of the Gaussian noise to be generated
  - mean range: float [-1.0, 1.0]
  - standard deviation range: float [0.0, 1.0]
    - currently the maximum allowable HW standard deviation is 0.144. The range is scaled by that maximum. EG. specifying 1.0 will produce data with a std of 0.144. Specifying 0.5 will produce data with a std of 0.144*0.5 = 0.072.
lstv_seconds: Long Sequence Test Vector length (in seconds). Must be greater than 0. Also must be an integer number of seconds for usage of BITE k values of form k = 56x + 16 (please see sample_rate_k)
tone_gens: generates a sinusoidal signal or ‘tone’ in the data that shows up as a spike, with configurable height (magnitude/’scale’) and width (in frequency)
- Empty array ([]) for no tones or max 1 element.
- Each tone gen element requires the following parameters:
  - description: string description of tone
  - pol_x and pol_y: the x and y polarization components of the tone, each of which requires:
    - frequency: the frequency of the sinusoid, range [0.0, 3.96e9]
    - scale: the magnitude of the signal, range [0.0,1.0]
    - Note: frequency and magnitude/scale are converted into values that can be written to the registers of the “bite_tone_gen” IP block: the magnitude is scaled into an unsigned 16-bit format, and the frequency is normalized with respect to the sampling rate and is scaled into signed 32-bit format (and is thus written to the “phase_inc” register of the block).
- Generated tones skip delay tracking in BITE, so delay models and tones may not work to typical expectation

Note about DDR4 Memory Allocation

The “lstv_seconds” and “sample_rate_k” will be used to determine the allocation of DDR4 memory required for storing the LSTV. Neither of these parameters is given a maximum value because they are constrained by the maximum size of the LSTV, which is 256GB, and so their maximum values would be inversely proportional. You can have a lot of samples over a short time span, or a long sample with few samples, etc. As a (rough) sanity check, the product of the length of the LSTV (in seconds) and the final k-value-modified sample rate cannot exceed 85228257260 without exceeding memory constraints.

Filter Definitions

See examples in filters.json (link) within the cbf_input_data/bite_config_parameters folder.

The filters contain four items:

description: description of the filter
band_edges: list of frequencies (normalized to 1.98 GHz) that you want to specify the amplitude at.
- Include the edges at 0 and 1.0 to ensure full frequency range is specified or it may cause an error: freq must start with 0 and end with fs/2
band_gains: list of desired gains (normalized to 1) at each of the frequencies specified in the band_edges list
window: dictionary describing the window to be used
- see SciPy Window Types for the window definitions

The code will generate a 1024 tap FIR filter to best meet the criteria specified above. An overly complicated filter design may not be perfectly met.

Future Functionality

SPFRx / receptor configuration file (like bite_configs).

Delay Model Configuration

The delay model configuration for a given test will determine which delay model package is used. See delay_model_config.json for examples.

Each Test ID in tests.json has one delay_model_config. Each of these configurations contains a set of parameters to be used by the test, including:

a description (required): an overview of the delay model configuration
a source_file (required): the filename that contains the delay models to be published
a validity_period_override_sec (optional): the validity_period_sec for all delay models defined in source_file is overridden with this value
a publish_lead_time_sec (optional): for a given model, the amount of time prior to it’s start_validity_sec that it is published. If not specified, a default of 5 seconds is used.
an overwrite_start_validity_sec (optional): boolean value used to determine whether to overwrite the delay models’ start_validity_sec based on the time that the delay models formatting step happens.
an array of drop_delay_models (optional): the indexes (1-based) of delay models to drop from the delay models in source_file. These delay models will not be published to the TMC emulator. If a delay model is dropped, the start_validity_sec values for subsequent models will not be shifted, so dropping delay models will create gaps in the time range covered by a test

Note: MCS ignores repeated delay models in the subarray device so subsequent tests with the same delay model (unless modified to be unique, for example updating start time at runtime) may not work as expected.

Delay Models

See the delay model package files in the delay_model_package folder for examples based on the ska-schemas schema: ska-csp-delaymodel.html.

This diagram shows the flow between the MATLAB generated CSV files to delay model json files that are added to BITE: CSV to delay model json flow

link to vsdx file: (CSV to delay model json flow vsdx)

The diagram below shows the sequence of commands in the Correlation Delay Model tests: Delay Model tests sequence

The PST Delay Model tests work in the same way as the Correlation Delay Model tests, and follow the above diagram.

Note: The following link is to the miro board that contains the delay model test sequence diagrams, if any future updates are needed to the sequence please refer to this link: delay model sequence miro board

Converting delay models from CSV to JSON format

To convert delay models from CSV to JSON format, use the script delay_models_csv_to_json.py in delay_models_csv Background info: There is a tool for generating Geometric Delay Models (GDM) that outputs delay model packages in CSV format. This tool can be used to generate inputs for BITE and for the TMC Emulator. See delay_models_csv/T1 for examples of delay model data generated by this tool:

gdm_sig_gen contains input files that can be used by BITE for signal generation
gdm_sig_proc contains delay models formatted as per-receptor CSV files, which must be converted into JSON format before being published to the TMC Emulator.
dm_base_time.json contains a value called base_time. The base_time is an offset that is:
- Added to the t_start/t_stop times in the gdm_sig_proc CSV files by delay_models_csv_to_json.py when creating the JSON delay model package file which will be fed to the TMC Emulator.
- Important note: the base time of sequential tests must be configured so that the timestamps do not jump backwards in time. Currently that is done by having multiple delay model packages with varying basetimes. This may be improved in the future in CIP-3497.

Note that the file names in gdm_sig_gen and gdm_sig_proc use the following acronyms:

HODCP: High Order Delay Coefficient Polynomial
DC: Delay Center (of subarray)
SRC: (Sky) Source

To run the script:

Follow the steps in the system-tests README.md to configure your python virtual environment
cd .../ska-mid-cbf-system-tests
Run the script (update arguments as needed): python test_parameters/delay_models_csv/delay_models_csv_to_json.py --out-file-full-path=test_parameters/delay_model_package/dm_package_3.json --csv-dir-path=test_parameters/delay_models_csv/T1/gdm_sig_proc/ --dish-parameter-mapping=4_boards

Slim Configurations

The slim configuration files describe the physical connections formed by the slim meshes. The fs slim files describe the frequency slice (FS) transportation from the VCCs to the FSPs. The vis slim files describe the transportation of the visibilities output.

When running at the PSI or ITF, the desired slim files are specified as a variable in the gitlab pipeline.

When running at the KAPB, the slim files are pre-downloaded from vault and already exist in the cbf controller. Those files are copied from the cbf controller pod to the slim_config folder at runtime (and named fs_slim_kapb.yaml and vis_slim_kapb.yaml) as they are required by the DCT checkpoint to run (the test parameters from system tests are copied to signal verification and used by various checkpoints).

Scan Configurations

See configure_scan.json in the configure_scan folder for example scan configurations based on ska-schemas schema: ska-csp-configure.html

Note that there can only be a single IP specified for the output_host field.

HW Configurations

The hardware configuration files describe ip addresses of the hardware used in a given test.

When running at the PSI, the desired hardware configuration can be used to select what hardware to run on. These configurations must be updated whenever the PSI hardware configuration changes.

When running at the ITF, the default hardware configuration file is the hw_config_itf.yaml.

When running at the KAPB, the hw config file is downloaded from vault and already exists in the cbf controller. There is a copy of the current kapb configuration stored in hw_config_kapb.yaml that does not include the lru username and password. This is to enable pinging the hardware at various stages in conftest and also in the hw ping test without downloading the hw config file with the lru password.

If the hardware configuration at kapb changes, the hw_config_kapb.yaml file will need to be updated to match. That can be done by downloading the hw config file from the cbfcontroller pod (ds-cbfcontroller-controller-0:/app/mnt/hw_config/hw_config.yaml).

Mid.CBF InitSysParam

See init_sys_param.json in the test_parameters folder for example initsysparam parameters based on ska-schemas schema: ska-mid-cbf-initsysparam.html

Checkpoints

See checkpoints.json in the checkpoints folder for example checkpoints. These provide methods for users to investigate data and behaviour at different checkpoints along the signal chain, and evaluate test runs based on pass/fail criteria. See the AA0.5 Test Strategy for a description of the various checkpoints.

Visibilities checkpoint

Format

Note: it is preferred that you completely remove an unwanted optional parameter rather than specifying it false for cleaner results tables.

"visibilities": {
	"capture_duration": <(s) float number min 0, optional>,
	"capture_timeout": <(s) float number min 0, optional>,
	"criteria": {
		"zip_file_exists": <boolean, required>,
		"fraction_of_data": <boolean, required>,
		"number_of_timestamps": <boolean, required>,
		"number_of_expected_baselines": <boolean, required>,
		"expected_channels": <boolean, required>,
		"vis_XX_delay_ps": <str, optional>,
		"vis_YY_delay_ps": <str, optional>,
		"vis_XY_delay_ps": <str, optional>,
		"vis_YX_delay_ps": <str, optional>,
		"vis_XX_mean_delay_ns": <str, optional>,
		"vis_YY_mean_delay_ns": <str, optional>,
		"vis_XY_mean_delay_ns": <str, optional>,
		"vis_YX_mean_delay_ns": <str, optional>
	},
	"baselines_with_detailed_plots": <list of strings, optional, format [SKAxxx-SKAxxx] with lower receptor first (eg SKA036-SKA063, not SKA063-SKA036)>,
	"delay_plots": <boolean, optional>
}

Description

capture_duration: controls the length of time system-tests captures visibilities for in seconds, default in system tests is 5 seconds
- Note: for simultaneous CORR + PST capture, CORR must have a longer capture_duration than PST (CIP-4767)
capture_timeout: controls time allowed for processing (until timing out) after the EndScan command
criteria: includes pass/fail checks on the visibilities
- zip_file_exists: specify true to check whether visibilities were successfully saved to a zip file
- fraction_of_data: specify true to check whether the fraction of data is all 1 (fraction of data less than 1 indicates flagged data)
- number_of_timestamps: specify true to calculate whether the expected number of timestamps (based on the capture time) has been captured in the visibilities data
- number_of_expected_baselines: specify true to check whether the visibilities data contains correct number of expected baselines given the test setup (eg. 10 baselines for 4-receptor correlator)
- expected_channels: specify true to check whether the visibilities data contains the correct channels given the configure_scan parameters
- vis_XX/YY/XY/YX_delay_ps: a string in the format of “[min, max]” in picoseconds to specify the expected range of visibilities delay in XX/YY/XY/YX polarization. The max and min delay is compared against the allowed range, derived from REQ-1652.
  - Tone generation may interfere with delay checks, see here
- vis_XX/YY/XY/YX_mean_delay_ns: a string in the format of “[min, max]” in nanoseconds to specify the expected range of visibilities mean delay in XX/YY/XY/YX polarization. The mean value is calculated per baseline and per frequency slice (in order to identify any problems with individual receptors or FSPs), and the min and max mean value is compared against the allowed range.
  - Tone generation may interfere with delay checks, see here
baselines_with_detailed_plots: specify using receptor pairs which baselines should have a magnified plot of the data in the visibilities report. Note that currently the smaller receptor in the pair must be specified first, ie. SKA036-SKA100, not SKA100-SKA036.
delay_plots: specify true to plot delays. The delay plots shows the delay (measured by dividing the phase by sky frequency) against the frequency in MHz.

`tcpdump` and `vis.pcap`

In addition to the visibility_capture.zip file that is generated for the visitiblities that are captured, this checkpoint also runs tcpdump in parallel.

tcpdump listens to the fsp output_ports specified in configure_scan.json and copies all the UDP packets that are sent to the visibiltiy capture during this checkpoint and stores it in the vis.pcap file. This file can be open using Wireshark and requires decoding for the packets to be human readable.

vis.pcap is to be used for debuging and will still be generated even if visiblity capture fails to create the visibility_capture.zip and/or when zip_file_exists fails.

Both files can be found in the test artifacts on the python-test stage in the GitLab pipeline for the test under Artifacts\results\<test_id>\visibilities\

Identifying baselines

One aspect of the visibilities checkpoint is being able to identify the baselines. For tests with multiple baselines, the filters have been adjusted to give the data from each receptor a unique shape. For a 4-receptor Test_6 run, the expected output for a single FSP (200 MHz) is displayed below. The MANUAL INSPECTION instructions in the Test_6 description explain in more detail what to look out for, and read as follows:

Receptors SKA001, SKA036, and SKA063 should show evidence of 25, 50, and 100 MHz repeated pass and stop bands respectively, ie. for a 200MHz frequency slice they will appear as follows: SKA001: |^|__|^|__|^|__|^|__, SKA036: |^^|___|^^|___, SKA063: |^^^^|____. SKA100 is left with an all pass filter.

Test 6 visibilities magnitude plot with filters plotted for multiple baselines against frequency

Test 6 visibilities phase plot with filters plotted for multiple baselines against frequency

Further example images can be found in the example_images folder.

Gitlab Automated Pipeline Visibilities Redirection

Visibilities for the automated gitlab pipelines are redirected to a PVC located on pod artifacts-sts-0 in namespace system-tests-artifacts. Once a test has been executed on an automated Gitlab pipeline one can interact with the PVC using Kubernetes commands to access the visibilities data. The specific path to the relevant visibilities will be based on the pipeline id and job id and can be found in the test execution in Jira or in the main log of the python-test job.

The following commands can be used to access the relevant data, from a shell on the PSI:

Kubernetes commands View data folder/contents:

kubectl exec artifacts-sts-0 -n system-tests-artifacts – ls /app/artifacts/pline_<pipeline id>_job_<job id>

Copy data to local:

kubectl cp system-tests-artifacts/artifacts-sts-0:/app/artifacts/pline_<pipeline id>_job_<job id> <desired local dir>

Cleanup

An automated cronjob is run weekly to delete old data. The script copied to the PSI worker node to perform this duty is located at ska-mid-cbf-system-tests/scripts/cleanup_artifacts.sh and contains more information about the running process and related logging.

Pulsar Timing (PST) checkpoint

Format

"pst": {
	"capture_duration": <(s) float number min 0, optional>,
	"capture_timeout": <(s) float number min 0, optional>,
	"criteria": {
		"pst_file_exists": <boolean, required>,
		"relative_output_weight": <boolean, required>,
		"expected_channels": <boolean, required>,
		"data_integrity_checks": {
			"number_of_timestamps": <boolean, optional>,
			"per_channel_data_continuity": <boolean, optional>,
			"timestamp_start_window_limit_s": <float, optional>,
			"timestamp_end_window_limit_s": <float, optional>
		}
	}
}

Description

capture_duration: controls the length of time system-tests captures pulsar timing for in seconds, default in system tests is 0.25s
- Note: for simultaneous CORR + PST capture, CORR must have a longer capture_duration than PST (CIP-4767)
capture_timeout: controls time allowed for processing (until timing out) after the EndScan command
criteria: includes pass/fail checks on the pulsar timing data
- pst_file_exists: specify true to check whether pulsar timing data was successfully saved to a pkl file
- relative_output_weight: specify true to check whether the relative output weight is 1.0. Relative output weight in the packets are a value between 0.0 and 1.0 - depending on how many of the configured receptors (in the subarray) contributed to the beam (the number of contributing receptors divided by receptors in the subarray). It is a measure of quality.
- expected_channels: specify true to check whether the pulsar timing data contains the correct channels given the configure_scan parameters
- data_integrity_checks: includes pass/fail checks on pst data count, continuity, and timing
  - number_of_timestamps: specify true to calculate whether the expected number of unique timestamps (based on the capture time) has been captured in the pulsar timing data
    - Note: number_of_timestamps does not check the channel population of each timestamp. timestamp_end_window_limit_s, timestamp_start_window_limit_s, and per_channel_data_continuity must all be used in addition to number_of_timestampsto ensure the completeness of timestamps.
  - per_channel_data_continuity: specify true to check whether each channel has continuous data flow from its start timestamp to its end timestamp (specifically checks if the channel’s number of unique timestamps matches an expected number from calculating using the time window between start and stop timestsamp and the pst data rate)
  - timestamp_start_window_limit_s: float representing a time window, with size seconds, which checks if start timestamps of all channels are within said time window, relative to one another
  - timestamp_end_window_limit_s: float representing a time window, with size seconds, which checks if end timestamps of all channels are within said time window, relative to one another

Please note that the automated PST test here does not include all the checkpoints. In order to perform diagnosis you can run the pst notebooks in signal verification repo. Instructions as follows:

Navigate to signal verification repo and notebooks/cbf_notebook
Edit test_config.yaml, assign_resources.json, and configure_scan.json as needed
Run 1_mcs_setup.ipynb, 2_bite_setup.ipynb, and 3_run_scan.ipynb respectively to start the scan
You can look into 3_run_scan.ipynb and 4_cbf_diagnosis.ipynb to run any desired checkpoints. Note that PST does not have all the checkpoints as visibilities. For PST aignal chain refer to the diagram below:

Checkpoints #10 (post channelizer histogram) and #12 (Ethernet) are the same as with the correlator bitstream. #11 (PST Sync Buffer) is a different component for PST but operates in a similar way to the correlator DDR corner turner component. Both are responsible for time synchronization of the data. They both have status registers to monitor the state to indicate if synchronization has been achieved and operating nominally.

You can run 5b_plot_pst.ipynb to plot pst data

Note: the instructions for running the notebooks are in the readme under the same directory as the notebooks.

Talon Status checkpoint

Format

"talon_status": {
	"talons": <list[str], required>
	"memory_blocks": <list[str], required>,
	"criteria": {
		"base_clock_frequency": <str, required>,
		"memory_block": {
			"emif_clock_frequency": <str, required>,
			"emif_pll_locked": <boolean, required>,
			"emif_local_reset_done": <boolean, required>,
			"emif_local_calibration_success": <boolean, required>,
			"emif_local_calibration_fail": <boolean, required>
		}
	}
}

Description

talons (required): a list of strings containing the talon target numbers on which the wideband state count checkpoint will be validated (e.g. [“001”, “002”])
memory_blocks (required): a list of strings. Valid values are “Bottom Left”, “Bottom Right”, and “Top Right”, corresponding to the installed DDR4 slots.
base_clock_frequency: a string in the format of “[min, max]” to specify the expected range of base clock frequency. The min and max should be integers.
emif_clock_frequency: a string in the format of “[min, max]” to specify the expected range of EMIF clock frequency. The min and max should be integers.
emif_pll_locked: boolean indicating if the PLL is locked. Should normally be true.
emif_local_reset_done: boolean indicating the state of the local_reset_done signal from the EMIF. Should normally be true.
emif_local_calibration_success: boolean indicating the state of the local_cal_success signal from the EMIF. Should normally be true.
emif_local_reset_done: boolean indicating the state of the local_reset_done signal from the EMIF. Should normally be true.

Wideband State Count checkpoint

Format

"wideband_state_count": {
	"talons": <list[str], required>
	"criteria": {
		"samples_accumulated": <str, required> 
	}
},

Description

talons (required): a list of strings containing the talon target numbers on which the wideband state count checkpoint will be validated (e.g. [“001”, “002”])
samples_accumulated: a string in format “(min, max)” to specify the range of expected number of samples accumulated by the wideband state count. The min and max should be integers. The max can be “infinity” if there is no upper bound.

Histogram checkpoints

Format

"histograms": [
	{
		"talon": <str, required>
		"inspection_points": <list[str], required>
		"criteria": {
			"capture_status": <boolean, required>
		}
	}
]

Description

talon (required): A string containing the talon target number on which the ddr4 corner turner checkpoint will be validated (e.g., “003”).
inspection_points (required): a list of inspection point IDs to be run on this talon board. The valid values are “E_PRE_VCC”, “E_POST_VCC_{x}”, “E_POST_RDT_{x}”, “E_POST_CH16K_{x}”.
- E_PRE_VCC: histogram showing the input data to VCC.
- E_POST_VCC_{x}: histogram showing the input data to a FSP. {x} is the zero-based VCC ID (ie. ‘0’ for VCC from 001). Despite its name, the data is not collected at the output of a VCC, but input of a FSP.
- E_POST_RDT_{x}: histogram showing the data after the resampler delay tracker. {x} is the zero-based VCC ID (ie. ‘0’ for VCC from 001). E_POST_RDT is not available if using bitstreams including and later than 1.0.0 see notes https://gitlab.com/ska-telescope/ska-mid-cbf-talondx/-/releases/ska-mid-cbf-talondx-v1.0.0
- E_POST_CH16K_{x}: histogram showing the data after the 16k fine channelizer. {x} is the zero-based VCC ID (ie. ‘0’ for VCC from 001).
capture_status (required): set to true if we want to check that capturing data is successful.

Packet Stream Repair checkpoint

Format

"packet_stream_repair": {
	"talon": <str, required>
	"validate_registers": <boolean, required>
}

Description

talon (required): A string containing the talon target number on which the packet stream repair checkpoint will be validated (e.g., “003”).
validate_registers: specify true to validate all packet stream repair device registers for the given talon against expected results - specify false to simply report out the packet stream repair device data for the given talon without pass/fail criteria.

DDR4 Corner Turner checkpoint

Format

"ddr4_corner_turner": {
	"talon": <str, required>
	"validate_registers": <boolean, required>
}

Description

talon (required): A string containing the talon target number on which the ddr4 corner turner checkpoint will be validated (e.g., “003”).
validate_registers: specify true to validate the ddr4 corner turner antenna_status and under_run_status for the given talon against expected results - specify false to simply report out the ddr4 corner turner device data for the given talon without pass/fail criteria.
antenna_status: represents whether data is flowing to each receptor. When the antenna_status is false for a receptor, that means that there is no data flowing to that receptor.
under_run_status: set to 1 for a receptor if the corner turner read timestamp is ahead of the write timestamp for that receptor, and 0 if not. The under_run_status is only checked if the antenna_status is true for that receptor.

In the DDR4 Corner Turner report, antenna_status and under_run_status should be read from right-to-left, with the right-most value representing the first receptor, which will correspond with the first entries in the criteria table, antenna_status_0 and under_run_status_0.

Resampler Delay Tracker checkpoint

Format

"resampler_delay_tracker": {
	"talon": <str, required>
	"validate_registers": <boolean, required>
}

Description

talon (required): A string containing the talon target number on which the resampler delay tracker checkpoint will be validated (e.g., “003”).
validate_registers: specify true to validate the fsp-corr-controller healthState attribute (if the corresponding FSP is configured for the scan) - specify false to simply report out the fsp-corr-controller device data for the given talon without pass/fail criteria. If validate_registers is true but the talon’s FSP is not configured in scan configuration, the fsp-corr-controller healthState is not validated.

Notes

The fsp-corr-controller healthState used in this checkpoint monitors both resampler delay tracker and DDR4 corner turner device server attributes and will report as FAILED if any of the following RDT or DCT fsp-corr-controller checks fail. RDT checks:

rdt_state: Validates that the RDT device server is running.
timestamp_gap_detected: Validates that there are no timestamp gaps detected (flagged when the timestamp increments by more than 1 second).
input_buffer_overflowed: Validates that no overflow conditions were detected on the input buffer.
output_fifo_overflow_polx: Validates that no overflow conditions were detected on the output FIFO X polarisation. If true, this may indicate that the other polarisation processing is being slow.
output_fifo_overflow_poly: ‘Validates that no overflow conditions were detected on the output FIFO Y polarisation. If true, this may indicate that the other polarisation processing is being slow. DCT checks:
antenna_status: Validates that the receptor is actively writing to the Corner Turner memory.
under_run_status: Validates that the CT read timestamp is not ahead of the write timestamp for the receptor.
wr_stop_seen: Validates that the reverse flow control does not exceed buffering for this write lane.

Additionally, RDT checkpoint will fail if data collection occurs during or after EndScan. This can occur when many receptors are used, due to the higher receptor numbers causing RDT checkpoint data collection to happen at a later time in the scan sequence.

100G Ethernet checkpoint

Format

"one_hundred_g_ethernet": {
	"talon": <str, required>
	"validate_registers": <boolean, required>
}

Description

talon (required): A string containing the talon target number on which the 100G ethernet checkpoint will be validated (e.g., “003”).
validate_registers: specify true to validate the relevant 100G ethernet device attributes against expected results - specify false to simply report out the 100G ethernet device data for the given talon without pass/fail criteria.

AA0.5 implementation expected results:

100G ethernet interface receiving (Rx) and transmitting (Tx) BITE data

eth_0_data_flow_active: Expecting true on all talons
eth_0_has_data_error: Expecting false on all talons 100G ethernet interface receiving (Rx) and transmitting (Tx) visibility data
eth_1_data_flow_active: If multiple talon test, expecting true on talon 001 only. If single talon test, expecting true on corresponding talon. Else, expecting false.
eth_1_has_data_error: Expecting false on all talons.

Note: The 100G Ethernet attributes are currently captured from the signal-verification collector script between RDT and wideband state count collectors near the start of a scan. This timing is needed to accurately capture the data flowing at the eth0 and eth1 interfaces.

Identify Unused Parameters

For code cleanliness and to minimize unused parameters, we have a function that identifies any parameters that are not in use. Execute ./test_parameters.py with the --unused_params flag to utilize this function.

Test Parameters

Summary

Mapping

CBF Input Data

BITE Configuration

Note about DDR4 Memory Allocation

Filter Definitions

Future Functionality

Delay Model Configuration

Delay Models

Converting delay models from CSV to JSON format

Slim Configurations

Scan Configurations

HW Configurations

Mid.CBF InitSysParam

Checkpoints

Visibilities checkpoint

Format

Description

tcpdump and vis.pcap

Identifying baselines

Gitlab Automated Pipeline Visibilities Redirection

The following commands can be used to access the relevant data, from a shell on the PSI:

Kubernetes commands View data folder/contents:

Copy data to local:

Cleanup

Pulsar Timing (PST) checkpoint

Format

Description

Talon Status checkpoint

Format

Description

Wideband State Count checkpoint

Format

Description

Histogram checkpoints

Format

Description

Packet Stream Repair checkpoint

Format

Description

DDR4 Corner Turner checkpoint

Format

Description

Resampler Delay Tracker checkpoint

Format

Description

Notes

100G Ethernet checkpoint

Format

Description

AA0.5 implementation expected results:

Identify Unused Parameters

`tcpdump` and `vis.pcap`