LAG (Link Aggregation, port-channel)

IEEE 802.3ad link aggregation (also known as port channel) enables you to group Ethernet interfaces to form a single link layer interface, also known as a link aggregation group (LAG), for load-balancing or failover.

All the channelized ports need to be configured with a valid speed setting.

If the LACP fallback feature is enabled, one member port per port channel will be active during the fallback mode. The LACP Fallback Feature allows an active LACP interface to establish a Link Aggregation (LAG) before it receives LACP PDUs from its peer.

The “min-links” parameter sets the minimum number of active links required to bring up the LAG interface.

Notes:

  • LACP in SONiC comes from an open-source project – teamd. It follows IEEE802.3ad and supports only LACP for physical ports. Static aggregation is not supported.
  • No packets can be transmitted over a LAG interface before a unit receives LACP PDUs from its peer.
  • The LACP Fallback Feature allows an active LACP interface to establish a Link Aggregation (LAG) before receiving LACP PDUs from its peer. LACP fallback will run on the members of the lower port number. (It will only run on one member port).
  • LACP mode must be in SLOW mode for all LAG interfaces on a SONiC device. (https://github.com/Azure/SONiC/wiki/Fast-Reboot) SONiC LACP PDU interval is 30 seconds, and it’s not configurable.

Example model & SONiC version:

  • Aurora 615/715
  • Netberg SONiC: sonic-202012-nb-inno-211121

LAG configuration

 

sonic_lag_1

LAG members cannot have individual VLAN and IP addresses prior to LAG establishment.

Check the configuration:

admin@sonic:~$ show vlan brief
admin@sonic:~$ show ip interfaces
admin@sonic:~$ show interfaces status

Method 1

Create a PortChannel interface:

sudo config portchannel add PortChannel01

(Optional) LACP fallback = True configuration – by default, LACP fallback = False

Add members to the interface:

admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet0
admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet12

Save the settings:

admin@sonic:~$ sudo config save -y

Check the port channel status:

admin@sonic:~$ show interfaces portchannel
Flags: A - active, I - inactive, Up - up, Dw - Down, N/A - not available,
       S - selected, D - deselected, * - not synced
  No.  Team Dev       Protocol     Ports
-----  -------------  -----------  --------------------------
   01  PortChannel01  LACP(A)(Up)  Ethernet12(D) Ethernet0(D)

If there is N/A instead of port members – please add members again.

Check the interface status:

admin@sonic:~$ show interfaces status Ethernet0-12,PortChannel01
    Interface            Lanes    Speed    MTU    FEC       Alias           Vlan    Oper    Admin             Type    Asym PFC
-------------  ---------------  -------  -----  -----  ----------  -------------  ------  -------  ---------------  ----------
    Ethernet0  121,122,123,124     100G   9100     rs   Ethernet0  PortChannel01    up         up  QSFP28 or later         N/A
    Ethernet4  125,126,127,128     100G   9100     rs   Ethernet4         routed    down       up              N/A         N/A
    Ethernet8  113,114,115,116     100G   9100     rs   Ethernet8         routed    up         up              N/A         N/A
   Ethernet12  117,118,119,120     100G   9100     rs  Ethernet12  PortChannel01    down       up  QSFP28 or later         N/A
PortChannel01              N/A     200G   9100    N/A         N/A         routed    up         up              N/A         N/A

Method 2

Modify the PORTCHANNEL hierarchy statements in the /etc/sonic/config_db.json file.

    "PORTCHANNEL": {
        "PortChannel01": {
            "admin_status": "up",
            "mtu": "9100"
        }
    },
    "PORTCHANNEL_MEMBER": {
        "PortChannel01|Ethernet0": {},
        "PortChannel01|Ethernet12": {}
    },

sonic_lag_2

One physical link is disconnected.

Expected results:

  • With min-links=2 configuration, only one LAG member is up, and the PortChannel01 is down.
  • Only one LAG member is up without min-links configuration, and the PortChannel01 is up.

Create a port channel with min-links

admin@sonic:~$ sudo config portchannel add PortChannel01 --min-links=2

Add members:

admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet0
admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet12

Result:

admin@sonic:~$ show interfaces status Ethernet0-12,PortChannel01
    Interface            Lanes    Speed    MTU    FEC       Alias           Vlan    Oper    Admin             Type    Asym PFC
-------------  ---------------  -------  -----  -----  ----------  -------------  ------  -------  ---------------  ----------
    Ethernet0  121,122,123,124     100G   9100     rs   Ethernet0  PortChannel01    down       up  QSFP28 or later         N/A
    Ethernet4  125,126,127,128     100G   9100     rs   Ethernet4         routed    down       up              N/A         N/A
    Ethernet8  113,114,115,116     100G   9100     rs   Ethernet8         routed    up         up              N/A         N/A
   Ethernet12  117,118,119,120     100G   9100     rs  Ethernet12  PortChannel01    down       up  QSFP28 or later         N/A
PortChannel01              N/A     200G   9100    N/A         N/A         routed    down       up              N/A         N/A

Create a port channel without min-links

Remove the port channel first:

admin@sonic:~$ sudo config portchannel member del PortChannel01 Ethernet0
admin@sonic:~$ sudo config portchannel member del PortChannel01 Ethernet12
admin@sonic:~$ sudo config portchannel del PortChannel01

Create PortChannel1 without min-links configuration:

admin@sonic:~$ sudo config portchannel add PortChannel01
admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet0
admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet12

Result:

admin@sonic:~$ show interfaces status Ethernet0-12,PortChannel01
    Interface            Lanes    Speed    MTU    FEC       Alias           Vlan    Oper    Admin             Type    Asym PFC
-------------  ---------------  -------  -----  -----  ----------  -------------  ------  -------  ---------------  ----------
    Ethernet0  121,122,123,124     100G   9100     rs   Ethernet0  PortChannel01    down       up  QSFP28 or later         N/A
    Ethernet4  125,126,127,128     100G   9100     rs   Ethernet4         routed    down       up              N/A         N/A
    Ethernet8  113,114,115,116     100G   9100     rs   Ethernet8         routed    up         up              N/A         N/A
   Ethernet12  117,118,119,120     100G   9100     rs  Ethernet12  PortChannel01    down       up  QSFP28 or later         N/A
PortChannel01              N/A     200G   9100    N/A         N/A         routed    up         up              N/A         N/A

LAG with fallback

Limitations

LACP fallback mode may also kick in during the normal LACP negotiation process due to the timing, which might cause some unexpected traffic loss. For example, if the LACP PDUs sent by peer are dropped completely, local member port with fallback enabled may still enter fallback mode, which might end up with data traffic loss.

Create a port channel with fallback

admin@sonic:~$ sudo config portchannel add PortChannel02 --fallback=true
admin@sonic:~$ sudo config portchannel member add PortChannel02 Ethernet4
admin@sonic:~$ show interfaces portchannel
Flags: A - active, I - inactive, Up - up, Dw - Down, N/A - not available,
       S - selected, D - deselected, * - not synced
  No.  Team Dev       Protocol     Ports
-----  -------------  -----------  --------------------------
   01  PortChannel01  LACP(A)(Dw)  Ethernet12(D) Ethernet0(D)
   02  PortChannel02  LACP(A)(Dw)  Ethernet4(D)


NEWS

Latest news