1. Purpose[edit source]
This article describes how to configure TSN (Time-Sensitive Networking) on switch
This article provides different steps to execute TSN scenario with switch:
- Need synchronization across the network. TSN requires precise time synchronization to ensure that time-critical data is delivered on time and in the correct order.
- IEEE 802.1Qav
- IEEE 802.1Qbv
2. IEEE 802.1AS-Rev or gPTP synchronization[edit source]
Proprietary PTP stack used for gPTP synchronization, which is provided by TTTech
Deptp is started by default
Basic configuration of the PTP stack is done via configuration file, which has XML format. See /etc/deptp/ptp_config.xml for stack config This stack:
- Start gPTP protocol
- synchronize system time to PTP clock
PTP Management tool called deptp tool can be used to access PTP runtime data from Linux command line. Useful commands:
// Show offset from master watch -n1 deptp_tool --get-current-dataset // Show PTP state of sw0p2 deptp_tool -t1 --get-port-dataset // Show PTP state of sw0p3 deptp_tool -t2 --get-port-dataset
3. IEEE 802.1Qav[edit source]
Maximum 4 Queues supported by GMAC and maximum 8 Queues supported by switch
An example configuration with 4 reserved bandwidth across Switch given below:
- Uses 4 Queues (Q0-Q3). Each Q has a bandwidth reserved in h/w.
- Queue 0 (Yellow stream) has No Bandwidth Restriction, general traffic (Prio 0)
- Queue 1 (Blue stream) has 40% of Maximum bandwidth reserved (300Mbps) (Prio 2)
- Queue 2 (Green stream) has 30% of Maximum bandwidth reserved (200Mbps) (Prio 4)
- Queue 3 (Red stream) has 20% of Maximum bandwidth reserved (100Mbps) (Prio 6)
Q3 is the highest priority Queue and Q0 is the lowest priority
Here are the steps to configure this setup.
Bring up end1/sw0ep interface
ip link add link sw0ep name sw0ep.200 type vlan id 200 ip link set sw0ep.200 type vlan egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7 ifconfig sw0ep.200 192.168.200.60 netmask 255.255.255.0 bridge vlan add vid 200 dev sw0p1 bridge vlan add vid 200 dev sw0p2 bridge vlan add vid 200 dev sw0p3
CBS is used to enforce a Quality of Service by limiting the data rate of a traffic class, to separate packets into traffic classes the user may choose mqprio(8) https://man7.org/linux/man-pages/man8/tc-mqprio.8.html, and configure it like this:
Network core and Driver uses the skb priority to deliver frames to specific h/w queues. In the above case, priority 6 (packet) goes to Q3, priority 4 goes to Q2, priority 2 goes to Q1 and priority 0 SKB goes to Q0.
./set-cbs-idleslope.sh sw0p3 6 100000000
+ tsntool fqtss tsa 6 cbs sw0p3
+ tsntool fqtss slope 6 100000000 sw0p3
+ tsntool fqtss show sw0p3
Traffic Class Algorithm operIdleSlope
0 Strict prio n/a
1 Strict prio n/a
2 Strict prio n/a
3 Strict prio n/a
4 Strict prio n/a
5 Strict prio n/a
6 CBS 100000000 bps
7 Strict prio n/a
./set-cbs-idleslope.sh sw0p3 4 200000000
+ tsntool fqtss tsa 4 cbs sw0p3
+ tsntool fqtss slope 4 200000000 sw0p3
+ tsntool fqtss show sw0p3
Traffic Class Algorithm operIdleSlope
0 Strict prio n/a
1 Strict prio n/a
2 Strict prio n/a
3 Strict prio n/a
4 CBS 200000000 bps
5 Strict prio n/a
6 CBS 100000000 bps
7 Strict prio n/a
./set-cbs-idleslope.sh sw0p3 2 300000000
+ tsntool fqtss tsa 2 cbs sw0p3
+ tsntool fqtss slope 2 300000000 sw0p3
+ tsntool fqtss show sw0p3
Traffic Class Algorithm operIdleSlope
0 Strict prio n/a
1 Strict prio n/a
2 CBS 300000000 bps
3 Strict prio n/a
4 CBS 200000000 bps
5 Strict prio n/a
6 CBS 100000000 bps
7 Strict prio n/a
With set-cbs-idleslope.sh :
#!/bin/bash
#implemented for tsntool 2.4.0
if [[ "$#" -ne 3 ]] ; then
echo "Usage: $0 port traffic-class idleSlope"
exit 1
fi
port=$1
trafficClass=$2
idleSlope=$3
set -x
tsntool fqtss tsa $trafficClass cbs $port
tsntool fqtss slope $trafficClass $idleSlope $port
tsntool fqtss show $port
Run 4 iperf sessions to generate several streams with different priorities. Remote PC or other device connected to sw0p3 (ETH1) with IP address 192.168.200.30, and other device/PC connected to sw0p2 (ETH3) to launch 2 x iperf3 client to generate streams.
Run 4 iperf sessions to generate several streams with different priorities Remote PC or other device connected to sw0p3 with IP address 192.168.200.30
iperf3 -s -p 5003& iperf3 -s -p 5002& iperf3 -s -p 5001& iperf3 -s -p 5000&
Remote PC or other device connected to sw0p2 with IP address 192.168.200.40
//Stream with SKB priority 6 : TOS=0x14 == SO_PRIORITY=6 // https:man7.org/linux/man-pages/man8/tc-prio.8.html iperf3 -c 192.168.200.30 -u -b 0 -p 5003 -l1472 -t60 --tos 0x14 &
//Stream with SKB priority 4 iperf3 -c 192.168.200.30 -u -b 0 -p 5002 -l1472 -t60 --tos 0x1a &
At DUT, start transmission of stream using iperf3
//Stream with SKB priority 2 iperf3 -c 192.168.200.30 -u -b 0 -p 5001 -l1472 -t60 --tos 0xa &
//Stream with SKB priority 0 (Best effort) iperf3 -c 192.168.200.30 -u -b 0 -p 5000 -l1472 -t60 &
Result without set-cbs-idleslope.sh rule apply :
For Traffic with priority 6:
[ 5] 0.00-60.85 sec 2.65 GBytes 374 Mbits/sec 0.196 ms 437454/2371320 (18%) receiver
For Traffic with priority 4:
[ 5] 0.00-60.21 sec 2.61 GBytes 372 Mbits/sec 0.046 ms 421125/2321380 (18%) receiver
For Traffic with priority 2:
[ 5] 0.00-60.00 sec 518 MBytes 72.4 Mbits/sec 0.047 ms 1826344/2195179 (83%) receiver
For Traffic with priority 0 (Best effort):
[ 5] 0.00-60.00 sec 831 MBytes 116 Mbits/sec 0.013 ms 2355974/2948076 (80%) receiver
Without CBS TSN rules in place, multiple traffic streams may contend for network resources, causing congestion and possible packet loss. This can lead to poor network performance and lower quality of service for users. Furthermore, without CBS TSN rules, there no prioritization of important traffic, such as real-time audio or video streams, which can cause delays and interruptions in these applications. In summary, the absence of CBS TSN rules can create an unstable and undependable network environment.
Result with set-cbs-idleslope.sh rule apply:
For Queue 3 with priority 6 and with BW of 100Mbps:
[ 5] 0.00-56.00 sec 647 MBytes 97.0 Mbits/sec 0.108 ms 3766160/4227385 (89%) receiver
For Queue 2 with priority 4 and with BW of 200Mbps:
[ 5] 0.00-60.00 sec 1.35 GBytes 193 Mbits/sec 0.066 ms 3438066/4422599 (78%) receiver
For Queue 1 with priority 2 and with BW of 300Mbps:
[ 5] 0.00-60.00 sec 2.02 GBytes 290 Mbits/sec 0.041 ms 2994134/4470767 (67%) receiver
For Queue0 with priority 0 (Best effort):
[ 5] 0.00-60.00 sec 1.11 GBytes 131 Mbits/sec 0.006 ms 134299/1921970 (12%) receiver
When TSN rules are implemented, the network can provide guaranteed bandwidth allocation and prioritization for different traffic streams. CBS TSN rules allow for the allocation of credits to different traffic streams, which can be used to transmit data over the network. This ensures that critical traffic, such as real-time audio or video streams, receive the necessary bandwidth and are not affected by other traffic streams. CBS TSN rules also prevent congestion and packet loss by limiting the amount of data that can be transmitted by each traffic stream. This results in a more predictable and reliable network environment, with improved quality of service for users.
4. IEEE 802.1Qbv[edit source]
An example configuration with 4 streams across Switch given below: (same topology as described in previous chapter)
- Uses 4 Queues (Q0-Q3). Each Q has a schedule reserved in h/w.
- Queue 0 (Yellow stream) considered best effort which have gate open for 250usec (Prio 0)
- Queue 1 (Blue stream) which have gate open for 125usec (Prio 2)
- Queue 2 (Green stream) which have gate open for 125usec (Prio 4)
- Queue 3 (Red stream) which have gate open for 125usec (Prio 6)
Q3 is the highest priority Queue and Q0 is the lowest priority
Here are the steps to configure this schedule.
Bring up end1/sw0ep interface
ip link add link sw0ep name sw0ep.200 type vlan id 200 ip link set sw0ep.200 type vlan egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7 ifconfig sw0ep.200 192.168.200.60 netmask 255.255.255.0 bridge vlan add vid 200 dev sw0p1 bridge vlan add vid 200 dev sw0p2 bridge vlan add vid 200 dev sw0p3
Example Qbv configuration: sgs is used for the configuration of Qbv. Save the following configuration to qbv.cfg.
with qbv.cfg:
sgs 125000 0x40 sgs 125000 0x10 sgs 125000 0x4 sgs 250000 0x1 sgs 250000 0 sgs 125000 0
This configuration sets
- set gate for traffic classes 6 for 125000 nanoseconds
- set gate for traffic classes 4 for 125000 nanoseconds
- set gate for traffic classes 2 for 125000 nanoseconds
- set gate for traffic classes 0 for 250000 nanoseconds
- all gates close for 375000 nanoseconds
Once the schedule is created, it should be written to the AdminControlList , before loading it to the OperCon- trolList. For writing it to the AdminControlList, call the tool tsntool as follows:
tsntool st wrcl sw0p3 qbv.cfg
To check that the configuration was written to the AdminControlList, the AdminControlList can be read as follows: tsntool st rdacl <interface> Example of printing to the terminal:
tsntool st rdacl sw0p3 sgs 125000 0x40 sgs 125000 0x10 sgs 125000 0x04 sgs 250000 0x01 sgs 250000 0x00 sgs 125000 0x00
To load this schedule to the OperControlList so that it gets taken into use, call this command:
tsntool st configure <basetime> <cycletime> <cycletime-extension> <interface>
The basetime is formatted as <seconds>.<nanoseconds>, as the current time of the switch. Note: The current
time of the switch can be read as follows:
cat /sys/class/net/sw0p3/ieee8021ST/CurrentTime
The parameter cycletime is formatted as <numerator>/<denominator> in seconds.
The cycletime-extension is formatted as plain nanoseconds. Note: The cycletime-extension parameter has to be
specified for legacy reasons. It is ignored by the driver.
An alternative is to use a relative time to apply the configuration. It’s in the same format, but with a ’+’ or a
’-’ just before.
tsntool st configure 0.0 1/1000 0 sw0p3
A cycle time of 1/1000 seconds = 1 millisecond is a lot longer than the time needed to complete one schedule
run-through. After the schedule has finished within a cycle, the gates remain in the last configured state until
the end of the cycle.
After calling the above commands, all the necessary parameters are set to have a basic Qbv configuration up
and running. It can be confirmed by running a command:
tsntool st show sw0p3 GatesEnabled: YES ConfigPending: NO ConfigChangeError: 1 AdminCycleTime: 1/1000 sec AdminCycleTimeExtension: 0 nsec AdminControlListLength: 6 entries OperCycleTime: 1/1000 sec OperCycleTimeExtension: 0 nsec OperControlListLength: 6 entries CurrentTime (TAI): 2023-03-03 10:41:46.737801550 (1677840106.737801550) AdminBaseTime (TAI): 1970-01-01 00:00:00.000000000 OperBaseTime (TAI): 1970-01-01 00:00:00.000000000 ConfigChangeTime (TAI): 2023-03-03 10:00:04.278000000 SupportedCycleMax: 335211525/1000000000 sec SupportedIntervalMax: 327675 nsec SupportedListMax: 1023 entries TickGranularity: 100000 1/10 nsec portNumTrafficClasses: 8 queueMaxSDUTable: TrafficClass queueMaxSDU TransmissionOverruns 0 1506 6961 1 1506 6961 2 1506 6961 3 1506 6961 4 1506 6961 5 1506 6961 6 1506 6961 7 1506 6961
Run 4 iperf sessions to generate several streams with different priorities. Remote PC or other device connected to sw0p3 (ETH1) with IP address 192.168.200.30, and other device/PC connected to sw0p2 (ETH3) to launch 2 x iperf3 client to generate streams.
Run 4 iperf sessions to generate several streams with different priorities Remote PC or other device connected to sw0p3 with IP address 192.168.200.30
iperf3 -s -p 5003& iperf3 -s -p 5002& iperf3 -s -p 5001& iperf3 -s -p 5000&
Remote PC or other device connected to sw0p2 with IP address 192.168.200.40
//Stream with SKB priority 6 : TOS=0x14 == SO_PRIORITY=6 // https:man7.org/linux/man-pages/man8/tc-prio.8.html iperf3 -c 192.168.200.30 -u -b 0 -p 5003 -l1472 -t60 --tos 0x14 &
//Stream with SKB priority 4 iperf3 -c 192.168.200.30 -u -b 0 -p 5002 -l1472 -t60 --tos 0x1a &
At DUT, start transmission of stream using iperf3
//Stream with SKB priority 2 iperf3 -c 192.168.200.30 -u -b 0 -p 5001 -l1472 -t60 --tos 0xa &
//Stream with SKB priority 0 (Best effort) iperf3 -c 192.168.200.30 -u -b 0 -p 5000 -l1472 -t60 &
Result without tsntool rule apply :
Result with tsntool rule apply:
5. References[edit source]