LAG (Link Aggregation, port-channel)
IEEE 802.3ad link aggregation (also known as port channel) enables you to group Ethernet interfaces to form a single link layer interface, also known as a link aggregation group (LAG), for load-balancing or failover.
All the channelized ports need to be configured with a valid speed setting.
If the LACP fallback feature is enabled, one member port per port channel will be active during the fallback mode. The LACP Fallback Feature allows an active LACP interface to establish a Link Aggregation (LAG) before it receives LACP PDUs from its peer.
The “min-links” parameter sets the minimum number of active links required to bring up the LAG interface.
Notes:
- LACP in SONiC comes from an open-source project – teamd. It follows IEEE802.3ad and supports only LACP for physical ports. Static aggregation is not supported.
- No packets can be transmitted over a LAG interface before a unit receives LACP PDUs from its peer.
- The LACP Fallback Feature allows an active LACP interface to establish a Link Aggregation (LAG) before receiving LACP PDUs from its peer. LACP fallback will run on the members of the lower port number. (It will only run on one member port).
- LACP mode must be in SLOW mode for all LAG interfaces on a SONiC device. (https://github.com/Azure/SONiC/wiki/Fast-Reboot) SONiC LACP PDU interval is 30 seconds, and it’s not configurable.
Example model & SONiC version:
- Aurora 615/715
- Netberg SONiC: sonic-202012-nb-inno-211121
LAG configuration
LAG members cannot have individual VLAN and IP addresses prior to LAG establishment.
Check the configuration:
admin@sonic:~$ show vlan brief admin@sonic:~$ show ip interfaces admin@sonic:~$ show interfaces status
Method 1
Create a PortChannel interface:
sudo config portchannel add PortChannel01
(Optional) LACP fallback = True configuration – by default, LACP fallback = False
Add members to the interface:
admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet0 admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet12
Save the settings:
admin@sonic:~$ sudo config save -y
Check the port channel status:
admin@sonic:~$ show interfaces portchannel Flags: A - active, I - inactive, Up - up, Dw - Down, N/A - not available, S - selected, D - deselected, * - not synced No. Team Dev Protocol Ports ----- ------------- ----------- -------------------------- 01 PortChannel01 LACP(A)(Up) Ethernet12(D) Ethernet0(D)
If there is N/A instead of port members – please add members again.
Check the interface status:
admin@sonic:~$ show interfaces status Ethernet0-12,PortChannel01 Interface Lanes Speed MTU FEC Alias Vlan Oper Admin Type Asym PFC ------------- --------------- ------- ----- ----- ---------- ------------- ------ ------- --------------- ---------- Ethernet0 121,122,123,124 100G 9100 rs Ethernet0 PortChannel01 up up QSFP28 or later N/A Ethernet4 125,126,127,128 100G 9100 rs Ethernet4 routed down up N/A N/A Ethernet8 113,114,115,116 100G 9100 rs Ethernet8 routed up up N/A N/A Ethernet12 117,118,119,120 100G 9100 rs Ethernet12 PortChannel01 down up QSFP28 or later N/A PortChannel01 N/A 200G 9100 N/A N/A routed up up N/A N/A
Method 2
Modify the PORTCHANNEL hierarchy statements in the /etc/sonic/config_db.json file.
"PORTCHANNEL": {
"PortChannel01": {
"admin_status": "up",
"mtu": "9100"
}
},
"PORTCHANNEL_MEMBER": {
"PortChannel01|Ethernet0": {},
"PortChannel01|Ethernet12": {}
},
LAG with min-links
One physical link is disconnected.
Expected results:
- With min-links=2 configuration, only one LAG member is up, and the PortChannel01 is down.
- Only one LAG member is up without min-links configuration, and the PortChannel01 is up.
Create a port channel with min-links
admin@sonic:~$ sudo config portchannel add PortChannel01 --min-links=2
Add members:
admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet0 admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet12
Result:
admin@sonic:~$ show interfaces status Ethernet0-12,PortChannel01 Interface Lanes Speed MTU FEC Alias Vlan Oper Admin Type Asym PFC ------------- --------------- ------- ----- ----- ---------- ------------- ------ ------- --------------- ---------- Ethernet0 121,122,123,124 100G 9100 rs Ethernet0 PortChannel01 down up QSFP28 or later N/A Ethernet4 125,126,127,128 100G 9100 rs Ethernet4 routed down up N/A N/A Ethernet8 113,114,115,116 100G 9100 rs Ethernet8 routed up up N/A N/A Ethernet12 117,118,119,120 100G 9100 rs Ethernet12 PortChannel01 down up QSFP28 or later N/A PortChannel01 N/A 200G 9100 N/A N/A routed down up N/A N/A
Create a port channel without min-links
Remove the port channel first:
admin@sonic:~$ sudo config portchannel member del PortChannel01 Ethernet0 admin@sonic:~$ sudo config portchannel member del PortChannel01 Ethernet12 admin@sonic:~$ sudo config portchannel del PortChannel01
Create PortChannel1 without min-links configuration:
admin@sonic:~$ sudo config portchannel add PortChannel01 admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet0 admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet12
Result:
admin@sonic:~$ show interfaces status Ethernet0-12,PortChannel01 Interface Lanes Speed MTU FEC Alias Vlan Oper Admin Type Asym PFC ------------- --------------- ------- ----- ----- ---------- ------------- ------ ------- --------------- ---------- Ethernet0 121,122,123,124 100G 9100 rs Ethernet0 PortChannel01 down up QSFP28 or later N/A Ethernet4 125,126,127,128 100G 9100 rs Ethernet4 routed down up N/A N/A Ethernet8 113,114,115,116 100G 9100 rs Ethernet8 routed up up N/A N/A Ethernet12 117,118,119,120 100G 9100 rs Ethernet12 PortChannel01 down up QSFP28 or later N/A PortChannel01 N/A 200G 9100 N/A N/A routed up up N/A N/A
LAG with fallback
Limitations
LACP fallback mode may also kick in during the normal LACP negotiation process due to the timing, which might cause some unexpected traffic loss. For example, if the LACP PDUs sent by peer are dropped completely, local member port with fallback enabled may still enter fallback mode, which might end up with data traffic loss.
Create a port channel with fallback
admin@sonic:~$ sudo config portchannel add PortChannel02 --fallback=true admin@sonic:~$ sudo config portchannel member add PortChannel02 Ethernet4 admin@sonic:~$ show interfaces portchannel Flags: A - active, I - inactive, Up - up, Dw - Down, N/A - not available, S - selected, D - deselected, * - not synced No. Team Dev Protocol Ports ----- ------------- ----------- -------------------------- 01 PortChannel01 LACP(A)(Dw) Ethernet12(D) Ethernet0(D) 02 PortChannel02 LACP(A)(Dw) Ethernet4(D)
