Station Configuration YAML

A station is described by a station configuration YAML. This YAML is extended from the same file used for AAVS-System, the software to monitor and control the Aperture Array Verification Systems (AAVS).

Below is an example:

observation:
    start_frequency_channel: 156.25e6
    bandwidth: 6.25e6

# Station configuration parameters
station:
    id: 0
    channel_truncation: 5
    beam_integration_time: 1
    channel_integration_time: 1
    beamformer_scaling: 4
    bitfile: "/opt/sps/bitfiles/tpm_firmware_6_2_0.bit"
    equalize_preadu: No
    pps_delays: [0, 0]
    adc_fullscale_voltage: 1.59

# Network configuration parameters
network:
    tile_40g_subnet: "10.192.0.1/25" # First Tile 40G IP / Subnet
    active_40g_qsfp: "port1-only"
    lmc:
        lmc_mac: None
        lmc_ip: "10.192.0.3"
        lmc_port: 4660
        tpm_cpld_port: 10000
        use_teng: True
        integrated_data_ip: "10.192.0.3"
        integrated_data_port: 4660
        use_teng_integrated: True

    csp_ingest:
        dst_mac: None
        dst_ip: "10.192.0.3"
        dst_port: 4660

# List of Subrack IPs (or names), and TPM slots which will form station
subracks:
    subrack_1:
        cpu_ip: "10.192.0.24"
        slots: [1, 3]

Tile Processing Modules (TPM) Configuration

To specify the TPMs you want to use, you specify each of their IP address on the NSDN (1GbE) network. Names can also be provided, however, this requires that a working DNS is available and configured to provide host name translation.

The format is:

tiles:
    - <TPM 1 ip address or name>
    - <TPM 2 ip address or name>

For example:

tiles:
    - "10.0.10.34"
    - "10.0.10.57"
    - tpm-3

Alternatively, you can specify the TPMs by referencing the subrack they are installed in. This allows more advanced functionality such as power control of the TPMs via the subrack management board CPU.

With this approach, TPM IP addresses are calculated based on the subrack management board CPU IP address and the slot number. As this is deterministic.

To specify the TPMs you want to use, you must specify the subrack CPU IP and the populated subrack slots you want to use. This is done in the following way:

subracks:
    <subrack name>:
        cpu_ip:  "<subrack cpu ip>"
        slots: <list of slots of tpms you want to use>

For example:

subracks:
    my_subrack_1:
        cpu_ip: "10.192.0.24"
        slots: [1, 3, 8]

    my_subrack_2:
        cpu_ip: "10.192.0.74"
        slots: [1, 2, 3, 4, 5, 6, 7]

NOTE: Both the tiles and subracks keys can exist in the same YAML file, however, the subracks configuration will always override the tiles section.

FPGA Firmware

Many historical TPM FPGA Firmware releases are automatically downloaded from the SKAO Central Artefact Repository as part of the installation. These are downloaded to a bitfiles folder in the root of the SPS Test Suite and symbolically linked to the install location. These follow nomenclature corresponding to firmware release X.Y.Z. i.e tpm_firmware_6_2_0.bit being version 6.2.0.

See the firmware release notes for more information TPM FPGA Firmware - Releases

To set the bitfile use the following:

station:
    bitfile: "<Absolute path to bitfile>"

For example:

station:
    bitfile: "/opt/sps/bitfiles/tpm_firmware_6_2_0.bit"

ADC Configuration

The full-scale input range of the ADCs on a TPM can be configured to any of the following available options:

1.36 V
1.47 V
1.59 V (default)
1.70 V
1.81 V
1.93 V
2.04 V

By default, all ADCs are initialised with a full-scale input voltage of 1.59 V.

To specify a different voltage, use the following:

station:
    adc_fullscale_voltage: <p-p full-scale voltage value>

For example:

station:
    adc_fullscale_voltage: 1.81

Network Configuration

Destination Configuration

There are currently 3 possible destinations for TPM data products; LMC, LMC integrated, and CSP.

The destination IP address and UDP port for each of these can be specified in the station configuration YAML:

network:
    lmc:
        lmc_ip: "<lmc destination IP>"
        lmc_port: <lmc destination port>
        integrated_data_ip: "<lmc integrated destination IP"
        integrated_data_port: <lmc integrated destination port>
    csp_ingest:
        dst_ip: "<csp destination IP>"
        dst_port: <csp destination port>

For example:

network:
    lmc:
        lmc_ip: "10.192.0.3"
        lmc_port: 5660
        integrated_data_ip: "10.192.0.3"
        integrated_data_port: 5660
    csp_ingest:
        dst_ip: "10.192.0.4"
        dst_port: 4660

LMC data products will be sent to the LMC destination [1]. These include:

Raw ADC data
Synchronized raw ADC data
Continuous channelized data
Burst channelized data
Tile beam data
Antenna buffer data

LMC integrated data products will be sent to the LMC integrated destination [1]. Typically these are used for diagnostics e.g antenna bandpass plots. These include:

Integrated channelized data
Integrated tile beam data

CSP data products will be sent to the CSP destination. These include:

Station beam data

LMC and LMC integrated data can either be routed via the NSDN or SDN, using 1GbE and 40GbE respectively.

For LMC data the Science Data Network (SDN) is enabled by setting the use_teng key to True. If False, the packets will be transmitted via the Non-Science Data Network (NSDN). The same is true for LMC integrated data, however using the use_teng_integrated key.

The naming of these keys is admittedly outdated from when the TPM supported 10GbE not 40GbE and should be refactored.

For example, routing LMC data products over the 40GbE Science Data Network, while routing LMC integrated data products over the 1GbE Non-Science Data Network:

network:
    lmc:
        lmc_ip: "10.192.0.3"
        lmc_port: 5660
        use_teng: True
        integrated_data_ip: "10.192.0.3"
        integrated_data_port: 5660
        use_teng_integrated: False
    csp_ingest:
        dst_ip: "10.192.0.4"
        dst_port: 4660

Source Configuration

The source IP addresses for the TPMs on the Non-Science Data Network, as mentioned above, is provided in either the tiles or subracks section of the YAML. The Science Data Network (40GbE) IP addresses, gateway and netmask are configurable via the tile_40g_subnet.

A network interface should be provided with the following syntax:

network:
    tile_40g_subnet: "<first-address>/<cidr>"

The IP addresses of the 40GbE link for each TPM in the station are dynamically allocated starting at the first address specified. Two IP addresses are provided two each TPM for its two QSFP connections, this is true even if only one is in use.

The gateway address is automatically calculated as the last IP address in the subnet (broadcast -1).

For example:

network:
    tile_40g_subnet: "10.130.0.1/25"

This specifies that TPMs should be allocated addresses incrementing from 10.130.0.1 with subnet 255.255.255.128 and gateway address 10.130.0.126. If the station had 2 TPMs, the first would be allocated 10.130.0.1 and 10.130.0.2, and the second 10.130.0.3 and 10.130.0.4.

NOTE: As IP addresses are dynamically assigned based on a starting address, care should be taken to ensure that IP addresses do not overlap with other stations. This is especially true if multiple users are using subsets of TPMs within subracks like at STFC-RAL.

If the tile_40g_subnet is not provided, then a default IP allocation will be used. This corresponds to the “AAVS” style IP allocation that follows this convention:

A /16 subnet (255.255.0.0) is used
Each TPM is allocated IP addresses derived from its NSDN address. These are of the format 10.0.1.X for the first QSFP and 10.0.2.X for the second QSFP.

For example two TPMs with NSDN (1GbE) IP addresses 10.132.6.45 and 10.132.6.46 would obtain SDN addresses 10.0.1.45, 10.0.2.45, 10.0.1.46 and 10.0.2.46.

The TPMs have 2 40GbE QSFP connections, most commonly only the lower connection (labelled port 1) is used. This can be controlled through the active_40g_qsfp section of the YAML which supports port1-only, port2-only and both-ports:

network:
    active_40g_qsfp: "port1-only"

For Cabinet Configuration YAML and Test Parameters YAML see the relevant pages.