1. Purpose[edit source]
This article describes how to configure TSN (Time-Sensitive Networking) endpoint
This article provides different steps to execute TSN scenario endpoint:
- Need synchronization across the network. TSN requires precise time synchronization to ensure that time-critical data is delivered on time and in the correct order.
- TSN 802.1Qav
- TSN 802.1Qbv
The choice between 802.1Qav and 802.1Qbv depends on the specific requirements of the application. 802.1Qav is typically used in applications where bandwidth requirements are variable and unpredictable, while 802.1Qbv is typically used in applications where bandwidth requirements are fixed and predictable.
2. IEEE 802.1AS-Rev or gPTP synchronization[edit source]
PTP is a protocol that enables the synchronization of clocks in a network. It uses a master-slave architecture, where one device acts as the master clock and the other devices act as slave clocks. The master clock sends synchronization messages to the slave clocks, which adjust their clocks to match the master clock.
Testing using ptp4l tool from linuxptp project
To check the ptp clock adjustment with PTP protocol, a PTP slave (client) and a PTP master (server) applications are needed to run on separate devices. Open source application package linuxptp can be used as slave and as well as master. Hence TX timestamp generation can be delayed (especially with low speed links) the ptp4l “tx_timestamp_timeout” parameter need to be set for ptp4l to work.
create file /etc/gPTP.cfg with content as below:
[global] gmCapable 1 priority1 248 priority2 248 logAnnounceInterval 0 logSyncInterval -3 syncReceiptTimeout 3 neighborPropDelayThresh 800 min_neighbor_prop_delay -20000000 assume_two_step 1 path_trace_enabled 1 follow_up_info 1 transportSpecific 0x1 ptp_dst_mac 01:80:C2:00:00:0E network_transport L2 delay_mechanism P2P
PTP synchro between two boards:
ptp4l -i end0 -p /dev/ptp0 -m -f /etc/gPTP.cfg
3. TSN 802.1Qav[edit source]
In TSN 802.1Qav or CBS (Credit-Based Shaper), each device is assigned a credit value that determines the amount of data it can transmit. The credit value is based on the bandwidth requirements of the device and the available bandwidth in the network. Devices with higher bandwidth requirements are assigned more credits, while devices with lower bandwidth requirements are assigned fewer credits.
As data is transmitted across the network, the credit values are decremented. When a device runs out of credits, it can no longer transmit data until it receives more credits. This ensures that time-critical traffic is given priority over other traffic, as devices with higher bandwidth requirements are able to transmit more data.
An example configuration with 4 reserved bandwidth given below:
Below commands set CBS with 100Mbit/s for queue 7 and
300Mbit/s for queue 6.
- Uses 4 Queues (Q0-Q3). Each Q has a bandwidth reserved in h/w. Maximum 4 Queues supported
- Queue 0 has No Bandwidth Restriction, general traffic
- Queue 1 has 10% of Maximum bandwidth reserved
- Queue 2 has 20% of Maximum bandwidth reserved
- Queue 3 has 30% of Maximum bandwidth reserved
- Q3 is the highest priority Queue and Q0 is the lowest priority
Here are the steps to configure this setup.
Bring up end0 interface
ip link add link end0 name end0.200 type vlan id 200 ip link set end0.200 type vlan egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7 ifconfig end0.200 192.168.200.60 netmask 255.255.255.0
CBS is used to enforce a Quality of Service by limiting the data rate of a traffic class, to separate packets into traffic classes the user may choose mqprio(8) https://man7.org/linux/man-pages/man8/tc-mqprio.8.html, and configure it like this:
tc qdisc add dev end0 clsact tc qdisc add dev end0 handle 100: parent root mqprio \ num_tc 4 \ map 0 0 1 1 2 2 3 3 3 3 3 3 3 3 3 3 \ queues 1@0 1@1 1@2 1@3 hw 0
Configure Bandwidth allocated for each queue (https://man7.org/linux/man-pages/man8/tc-cbs.8.html)
// Set bandwidth of queue 1 to be 100Mbps tc qdisc replace dev end0 parent 100:2 cbs \ idleslope 100000 \ sendslope -900000 \ hicredit 150 \ locredit -1350 \ offload 1
// Set bandwidth of queue 2 to be 200Mbps tc qdisc replace dev end0 parent 100:3 cbs \ idleslope 200000 \ sendslope -800000 \ hicredit 300 \ locredit -1200 \ offload 1
// Set bandwidth of queue 3 to be 300Mbps tc qdisc replace dev end0 parent 100:4 cbs \ idleslope 300000 \ sendslope -700000 \ hicredit 450 \ locredit -1050 \ offload 1
So for queue 0 the bandwidth should be around 400Mbps (port_transmit_rate - sum of all bandwidth of each queue)
sendslope = idleslope - port_transmit_rate
hicredit = max_interference_size * (idleslope / port_transmit_rate)
locredit = max_frame_size * (sendslope / port_transmit_rate)
with max_interference_size = max_frame_size = 1500
For more details: https://en.wikipedia.org/wiki/Time-Sensitive_Networking#IEEE_802.1Qav_Forwarding_and_queuing_enhancements_for_time-sensitive_streams
Network core and Driver uses the skb priority to deliver frames to specific h/w queues. In the above case, priority 6 to 15 SKB (packet) goes to Q3 (4th entry in map in the tc qdisc command), priority 4 to 5 SKB goes to Q2 (3rd entry in map) priority 2 to 3 SKB goes to Q1 (2nd entry in map) and priority 0 to 1 SKB goes to Q0 (1st entry in map)
Run 4 iperf sessions to generate several streams with different priorities Remote PC or other device connected to end0 with IP address 192.168.200.30
iperf3 -s -p 5003& iperf3 -s -p 5002& iperf3 -s -p 5001& iperf3 -s -p 5000&
At DUT, start transmission of stream using iperf3
//Stream with SKB priority 6 : TOS=0x14 == SO_PRIORITY=6 // https:man7.org/linux/man-pages/man8/tc-prio.8.html iperf3 -c 192.168.200.30 -u -b 0 -p 5003 -l1472 -t60 --tos 0x14 &
//Stream with SKB priority 4 iperf3 -c 192.168.200.30 -u -b 0 -p 5002 -l1472 -t60 --tos 0x1a &
//Stream with SKB priority 2 iperf3 -c 192.168.200.30 -u -b 0 -p 5001 -l1472 -t60 --tos 0xa &
//Stream with SKB priority 0 (Best effort) iperf3 -c 192.168.200.30 -u -b 0 -p 5000 -l1472 -t60 &
Result without tc rule apply :
For Prio 6:
[ 5] 0.00-60.00 sec 1.40 GBytes 200 Mbits/sec 0.046 ms 11841/1031880 (1.1%) receiver
For Prio 4: [ 5] 0.00-60.00 sec 40.1 MBytes 5.60 Mbits/sec 39.651 ms 936/29490 (3.2%) receiver
For Prio 2: [ 5] 0.00-60.00 sec 4.94 GBytes 707 Mbits/sec 0.035 ms 164701/3768910 (4.4%) receiver
For Prio 0 (Best effort): [ 5] 0.00-60.00 sec 106 MBytes 14.8 Mbits/sec 0.017 ms 1765/76956 (2.3%) receiver
Result with tc rule apply:
For Prio 6:
[ 5] 0.00-60.00 sec 1.72 GBytes 246 Mbits/sec 0.052 ms 189322/1444777 (13%) receiver
For Prio 4: [ 5] 0.00-60.00 sec 1.16 GBytes 167 Mbits/sec 0.064 ms 110342/959769 (11%) receiver
For Prio 2: [ 5] 0.00-60.00 sec 641 MBytes 89.6 Mbits/sec 0.266 ms 32740/489287 (6.7%) receiver
For Prio 0 (Best effort): [ 5] 0.00-60.00 sec 2.31 GBytes 331 Mbits/sec 0.006 ms 234299/1921970 (12%) receiver
4. TSN 802.1Qbv[edit source]
802.1Qbv (Time-Aware Shaper) is a traffic scheduling technique that uses time-based scheduling to ensure that time-critical traffic is given priority over other traffic. It assigns time slots to each device based on its priority level.
An example configuration with 4 schedule entries given below:
- Uses 4 Queues (Q0-Q3). Each Q has a Gate associated in h/w. Maximum 4 Queues/Gates supported
- 3 higher priority Gates open for 125usec (Q1, Q2 and Q3)
- Q3 is the higest priority Queue and Q0 is the lowest priority
- 1 lower priority (Q0) opens remaining gates for 250 usec
Here are the steps to configure this schedule.
Bring up end0 interface
ip link add link end0 name end0.200 type vlan id 200 ip link set end0.200 type vlan egress 0:0 1:1 2:2 3:3 4:4 5:5 6:6 7:7 ifconfig end0.200 192.168.200.60 netmask 255.255.255.0
Setup EST schedule with 4 Gates (Q0-Q3). For description of Command parameters, see manual page for taprio. https://man7.org/linux/man-pages/man8/tc-taprio.8.html TC0 <-> Q0, TC1 <-> Q1, TC2 <-> Q2 and TC3 <-> Q3
tc qdisc replace dev end0 parent root handle 100 taprio \ num_tc 4 \ map 0 0 1 1 2 2 3 3 3 3 3 3 3 3 3 3 \ queues 1@0 1@1 1@2 1@3 \ base-time 0000 \ sched-entry S 8 125000 \ sched-entry S 4 125000 \ sched-entry S 2 125000 \ sched-entry S 1 250000 \ sched-entry S 0 375000 \ flags 2
Where num_tc is same as number of queues = 4, maps 16 priorities to one of 4 TCs, queues specify the Queue associated with each TC, TC0 - One queue @0, TC1 - One queue @1, TC2 - One queue @2 and TC3 - One queue @3
sched-entry S 8 125000 S - SetGateStates operation 8 - Bit mask showing bit 4 set (Q3/TC3) 125000 - 125000 nsecs (125 usecs ) duration of Gate open The cycle-time is 1 msec (which is the sum of the intervals of individual schedule)
Enable classifier. Classifier is used to mark the packet based on packet meta data.
tc qdisc add dev end0 clsact
Network core and Driver uses the skb priority to deliver frames to specific h/w queues. In the above case, priority 6 to 15 SKB (packet) goes to Q3 (4th entry in map in the tc qdisc command), priority 4 to 5 SKB goes to Q2 (3rd entry in map) priority 2 to 3 SKB goes to Q1 (2nd entry in map) and priority 0 to 1 SKB goes to Q0 (1st entry in map)
Run 4 iperf sessions to generate several streams with different priorities Remote PC or other device connected to end0 with IP address 192.168.200.30
iperf3 -s -p 5003& iperf3 -s -p 5002& iperf3 -s -p 5001& iperf3 -s -p 5000&
At DUT, start transmission of stream using iperf3
//Stream with SKB priority 6 : TOS=0x14 == SO_PRIORITY=6 // https:man7.org/linux/man-pages/man8/tc-prio.8.html iperf3 -c 192.168.200.30 -u -b 0 -p 5003 -l1472 -t60 --tos 0x14 &
//Stream with SKB priority 4 iperf3 -c 192.168.200.30 -u -b 0 -p 5002 -l1472 -t60 --tos 0x1a &
//Stream with SKB priority 2 iperf3 -c 192.168.200.30 -u -b 0 -p 5001 -l1472 -t60 --tos 0xa &
//Stream with SKB priority 0 (Best effort) iperf3 -c 192.168.200.30 -u -b 0 -p 5000 -l1472 -t60 &
Result without tc rule apply :
Result with tc rule apply:
5. References[edit source]