Everflow and SPAN mirroring

Configuring mirroring in SONiC

Example model & SONiC version:

  • Aurora 615/715
  • Netberg SONiC: sonic-202012-nb-inno-211121

Everflow

Everflow is a network telemetry system that provides scalable and flexible access to packet-level information in large data centers. Everflow uses “match and mirror” functionality. Commodity switches can apply actions on packets that match flexible patterns over packet headers or payloads and then mirror packets to analysis servers by the action. This is an alternative to ERSPAN (Encapsulated Remote Switch Port Analysis).

Limitations:

  • Only ConfigDB editing, CLI commands are limited at the moment.
  • For mirror type, it doesn’t support the IPv6 destination setting on the ACL rule.
  • Mirror session becomes active status when the destination arp is learned on arp table.

Everflow configuration is consists of the following parts:

  • Session configuration. Contains information required to setup session.
  • ACL table configuration.
  • Traffic match configuration. Contains a list of ACL rules with action “mirror” which mean that matched traffic should be encapsulated by a mirror session.

Everflow can be enabled on certain production switches on-demand to debug live incidents. The “Topology for Everflow configuration example” figure shows the topology used for the configuration example. Everflow configuration can be carried out using the config_db.json on Netberg SONiC.

Figure 1. Topology for Everflow configuration example
Everflow topology

In the topology example, the traffic flows from Server1 (192.168.10.10) to Server2 (192.168.11.10), passing through switch Leaf1. The gateway IP address for Server1 configured on Leaf 1 is 192.168.10.250. The mirror session is applied on Leaf1 with the destination IP address of 10.0.0.2 and the GRE Tunnel type of 0x88be to the collector. The interface Ethernet0 on Leaf1 connects to Server1, while Ethernet4 connects to Server2. A route is added to the collector Linux VM for all the mirror traffic. The collected data is then used for filtering and visualization.

admin@sonic:~$ sudo config mirror_session add Mirror_Everflow 192.168.10.250 10.0.0.2 63 255
admin@sonic:~$ sudo nano /tmp/everflow_sample.json
{
   "ACL_RULE": {
       "MIRROR_ACL|Mirror_Rule": {
           "DST_IP": "192.168.11.10/24",
           "IP_PROTOCOL": "61",
           "MIRROR_ACTION": "Mirror_Everflow",
           "PRIORITY": "999",
           "SRC_IP": "192.168.10.10/24"
       }
   },
   "ACL_TABLE": {
       "MIRROR_ACL": {
           "policy_desc": "Mirror_ACLV4_CREATION",
           "ports": [
               "Ethernet0",
               "Ethernet4"
           ],
           "stage": "INGRESS",
           "type": "mirror"
       }
   },
       "MIRROR_SESSION": {
       "Mirror_Everflow": {
           "dscp": "63",
           "dst_ip": "10.0.0.2",
           "gre_type": "0x88be",
           "queue": "0",
           "src_ip": "192.168.10.250",
           "ttl": "255"
           }
     }
}
admin@sonic:~$ sudo config load /tmp/everflow_sample.json -y
Running command: /usr/local/bin/sonic-cfggen -j /tmp/everflow_sample.json --write-to-db
admin@sonic:~$ sudo config save -y
Running command: /usr/local/bin/sonic-cfggen -d --print-data > /etc/sonic/config_db.json

Results:

admin@sonic:~$ show mirror_session
ERSPAN Sessions
Name             Status    SRC IP          DST IP    GRE      DSCP    TTL  Queue    Policer    Monitor Port    SRC Port    Direction
---------------  --------  --------------  --------  -----  ------  -----  -------  ---------  --------------  ----------  -----------
Mirror_Everflow  active    192.168.10.250  10.0.0.2  0x88be     63    255  0
admin@sonic:~$ show acl table
Name        Type    Binding    Description            Stage
----------  ------  ---------  ---------------------  -------
MIRROR_ACL  mirror  Ethernet0  Mirror_ACLV4_CREATION  ingress
                    Ethernet4
admin@sonic:~$ show acl rule
Table       Rule         Priority    Action                           Match
----------  -----------  ----------  -------------------------------  ------------------------
MIRROR_ACL  Mirror_Rule  999         MIRROR INGRESS: Mirror_Everflow  DST_IP: 192.168.10.10/24
                                                                      IP_PROTOCOL: 61
                                                                      SRC_IP: 192.168.11.10/24

SPAN

Topology:

Figure 2. Topology for SPAN configuration example

span

The system configuration:

admin@sonic:~$ show vlan brief
+-----------+------------------+---------+----------------+-----------------------+-------------+
|   VLAN ID | IP Address       | Ports   | Port Tagging   | DHCP Helper Address   | Proxy ARP   |
+===========+==================+=========+================+=======================+=============+
|       100 | 192.168.100.1/24 |         |                |                       | disabled    |
+-----------+------------------+---------+----------------+-----------------------+-------------+
|       200 | 192.168.200.1/24 |         |                |                       | disabled    |
+-----------+------------------+---------+----------------+-----------------------+-------------+
admin@sonic:~$ show ip interfaces
Interface    Master    IPv4 address/mask    Admin/Oper    BGP Neighbor    Neighbor IP
-----------  --------  -------------------  ------------  --------------  -------------
Loopback0              10.1.0.1/32          up/up         N/A             N/A
Vlan100                192.168.100.1/24     up/up         N/A             N/A
Vlan200                192.168.200.1/24     up/up         N/A             N/A
docker0                240.127.1.1/24       up/down       N/A             N/A
eth0                   192.168.0.126/24     up/up         N/A             N/A
lo                     127.0.0.1/16         up/up         N/A             N/A

Add a mirror session:

sudo config mirror_session span add span_mirror Ethernet48 Ethernet40

Check the result:

admin@sonic:~$ show mirror_session
ERSPAN Sessions
Name             Status    SRC IP          DST IP    GRE      DSCP    TTL  Queue    Policer    Monitor Port    SRC Port    Direction
---------------  --------  --------------  --------  -----  ------  -----  -------  ---------  --------------  ----------  -----------

SPAN Sessions
Name         Status    DST Port    SRC Port    Direction    Queue    Policer
-----------  --------  ----------  ----------  -----------  -------  ---------
span_mirror  active    Ethernet48  Ethernet40  both
NEWS

Latest news