Application Notes for OF-DPA and Ryu SDN controller
This information is valid for Netberg Aurora 420, Aurora 620, Aurora 630, and Aurora 720 switches using OpenSwitch release 2.0.4 and above.
The spec referred below is the OpenFlow Data Plane Abstraction (OF-DPA): Abstract Switch Specification document.
This document is based on the real-world development of a telecom SDN product.
Q1: OF-DPA is updated with the BCM OFDPA experimental commands.
For OFDPA experimenter messages:
add-ofdpa-oam SWITCH OAM add oam described by OAM mod-ofdpa-oam SWITCH OAM modify specific oam del-ofdpa-oam SWITCH OAM delete matching OAM dump-ofdpa-oam SWITCH [OAM] print oam description add-ofdpa-drop-status SWITCH DROP add DROP described by DROP mod-ofdpa-drop-status SWITCH DROP modify specific DROP del-ofdpa-drop-status SWITCH DROP delete matching DROP
and so on.
Can you share some examples from them?
A1: Different chipsets can support different features in OF-DPA.
In TD2 and TH, they don’t support the OAM.
According to BCM release note, only Katana2 and Saber2 chipsets support this feature.
Q2: Cannot add an MPLS interface group.
ovs-ofctl add-group bridge_ofdpa group_id=0x9000000d,type=indirect,bucket=set_field:"00:00:00:00:00:11"-\>dl_src,set_field:"00:00:00:00:00:12"-\>dl_dst,set_field:5096-\>vlan_vid,group:0x3ea000d -O openflow13
Gives an error INVALID_GROUP.
A2: The group 0x3ea000d has to be created.
The vlan_vid value in the set_field of MPLS interface group has to be the same as the vlan_vid of group 0x3ea000d.
If the group 0x3ea000d was created, please modify your command as follows. Then the MPLS interface group can be created successfully.
Modify the set_field:5096-\>vlan_vid to set_field:5098-\>vlan_vid
ovs-ofctl add-group bridge_ofdpa group_id=0x9000000d,type=indirect,bucket=set_field:"00:00:00:00:00:11"-\>dl_src,set_field:"00:00:00:00:00:12"-\>dl_dst,set_field:5098-\>vlan_vid,group:0x3ea000d -O openflow13
Q3: According to the spec, MPLS_forwarding group ID 10 is supported.
This type includes A6. I.e., type 10(A) and subtype 6(6) – MPLS FastFailover group.
|
Type |
Subtype |
|
|---|---|---|
|
9 |
1 |
0x91xxxxxx MPLS L2 VPN |
|
9 |
12 |
0x9cxxxxxx Provider group |
|
9 |
13 |
0x9dxxxxxx Customer group |
|
9 |
14 |
0x9exxxxxx BUM group (used in VPLS pipeline) |
|
10 |
6 |
0xA6xxxxxx MPLS FF group |
We got an error “Invalid Group” when tried to use it.
Do you support this group?
A3: We support type 9 and subtype 1.
OFDPA 3.0.4.0 spec doesn’t support subtype 12, 13 and 14.
The type 10 and subtype 6 need the OAM feature, and the TD2 and TH doesn’t support OAM.
SPEC 3.10.3 MPLS-TP OAM Fast Failover Groups rely on OAM path fault detection for liveness. When OAM detects a fault on the worker path, the OAM connectivity verification process state machine will notify the network protection state machine, which in turn will invalidate the liveness property being monitored by the worker bucket. This then causes the Fast Failover Group to switch over to the protector path. The MPLS Fast Failover Group bucket “watch_port” parameter is configured to watch the operational status of an OAM Network Protection Logical Port defined for this purpose.
Q4: A VPWS script does not work as intended.
Cannot set a rule for port 12 in table 10 with an error Bad match/Bad field (frame 23 in the dump).
At the same time, no trouble for ports 13 and 14, where match rule is no different.
What are we doing wrong?
A4: The OFDPA experimenter is incorrect in your FLOW MOD message (frame 13). Please check your RYU about ofdpa experimenter parameters.
The Field 8 is MPLS L2 Port, and its length is the 32 bits, but your value is 0008000000c8, not 000000c8.
The Field 23 is MPLS Type, and its length is the 16 bits, but your value is 00170001, not 0001.
Action
Type: OFPAT_SET_FIELD (25)
Length: 24
OXM field
Class: OFPXMC_EXPERIMENTER (0xffff)
0001 000. = Field: 8
.... ...0 = Has mask: False
Length: 10
Experimenter: 0x00001018
Experimenter Value: 0008000000c8
Unknown OXM body.
[Expert Info (Note/Undecoded): Unknown OXM body.]
Action
Type: OFPAT_SET_FIELD (25)
Length: 16
OXM field
Class: OFPXMC_EXPERIMENTER (0xffff)
0010 111. = Field: 23
.... ...0 = Has mask: False
Length: 8
Experimenter: 0x00001018
Experimenter Value: 00170001
This flow can use ovs-ofctl to add it successfully.
ovs-ofctl -O OpenFlow13 add-flow bridge_ofdpa table=10,vlan_vid=0x10c8,in_port=12,actions=set_field:0x100c8-\>tunnel_id,set_field:0xc8-\>ofdpa_mpls_l2_port,set_field:1-\>ofdpa_mpls_type,goto_table:13
Q5: The same Bad match/Bad field error for table 13 (frame 24 in the dump).
We would expect the Bad action/Bad group error, because before that MPLS L2 VPN group wasn’t set in type 9 subtype 1 (frame 22 in the dump).
A5: The SPEC specifies the table 13 flow needs the Set-Field QoS Index action for the next table is table 16, but your flow doesn’t have this action.
– We checked your MPLS L2 VPN group (type 9 subtype 1) in the Group Mod message (frame 13), there is no mpls ttl action in your Group Mod. Please check the RYU about ofdpa experimenter.
Action
Type: OFPAT_SET_FIELD (25)
Length: 16
OXM field
Class: OFPXMC_NXM_1 (0x0001)
0011 110. = Field: 30
.... ...0 = Has mask: False
Length: 1
Unknown OXM body.
– This group can use ovs-ofctl to add it successfully.
ovs-ofctl -O OpenFlow13 add-group bridge_ofdpa group_id=0x910000c8,type=indirect,bucket=actions=ofdpa_push_l2hdr,push_vlan:0x8100,push_mpls:0x8847,set_field:10000-\>mpls_label,set_field:1-\>mpls_bos,set_mpls_ttl:240,ofdpa_push_cw,group:0x9300000e
Also, The RYU parser doesn’t follow OFDPA specification to encode the correct OFDPA experimenter match fields and actions.
Therefore, the error type is wrong, as well.
Q6: A problem to add a rule to the table 25 (supported in the spec, frame 26 in the dump) with the Bad match/Bad field error.
A6: The table 25 is read-only, it can’t add flow, only used to go to this table.
Q7: BCM experimenter parameter support?
mpls_l2_port (local, network) mpls_type (1 - VPWS, 2 - VPLS) ovid vrf PushL2Hdr PushCW PopCW
A7: We support the OFDPA experimenter.
You can use the below keywords in ovs-ofctl command:
ofdpa_mpls_l2_port ofdpa_mpls_type ofdpa_ovid ofdpa_vrf ofdpa_push_cw ofdpa_pop_cw ofdpa_push_l2hdr
Q8: Three rules, in OVS format they will be like:
ovs-ofctl add-flow bridge_ofdpa table=10,in_port=13,dl_vlan=1000,actions=goto_table:20 -O openflow13 ovs-ofctl add-flow bridge_ofdpa table=10,in_port=14,dl_vlan=1000,actions=goto_table:20 -O openflow13
ovs-ofctl add-flow bridge_ofdpa table=10,in_port=12,dl_vlan=200,actions=set_field:"0x100c8"-\>tunnel_id,set_field:"0xc8"-\>ofdpa_mpls_l2_port,set_field:"0x1"\>ofdpa_mpls_type,goto_table:20 -O openflow13
The first two rules are ok, the third fails with bad match/bad field error. This is strange, because it’s the same with the first two at the match section.
A8: The spec indicates this flow’s Goto-Table instruction should specify the MPLS L2 Flow Table(13), but your flow goes to table 20.
There are different flow type in table 10. The Goto-Table instruction should, according to your flow type to go to different next table.
Q9: [A5] The SPEC specifies the table 13 flow needs the Set-Field QoS Index action for the next table is table 16, but your flow doesn’t have this action.
Table 13 spec (MPLS L2 port) defines several pipelines (page 81 Table 39 MPLS L2 Port Flow Table Flow Entry Types).
qos_index is required only for MPLS PCP. If we understand this right – ingress mpls traffic with 802.1q tags (pcp marked).
VPSW does not have this requirement and allowed actions includes one of the mentioned tables in the goto instruction (Table 41 MPLS L2 Port Flow Table Instructions).
A9: Please refer to table 42 on page 83 of the spec.
Q10: [A8] The SPEC indicates this flow’s Goto-Table instruction should specify the MPLS L2 Flow Table(13), but your flow goes to table 20.
Yes, that’s right. But it’s a typo in the commands we typed. In the script it’s 13.
# add flow to table 10
match = parser.OFPMatch(in_port=in_port, vlan_vid=(0x1000 | cust_vid,0x1fff) )
actions = [ parser.OFPActionSetField(tunnel_id = (0x10000 | cust_vrf)),
parser.OFPActionSetField(ofdpa_oxm_mpls_l2_port=int(cust_vrf)),
parser.OFPActionSetField(ofdpa_oxm_mpls_type=1)
]
instrs = [
parser.OFPInstructionActions(ofproto.OFPIT_APPLY_ACTIONS, actions),
parser.OFPInstructionGotoTable(13)
]
flow_mod = parser.OFPFlowMod(cookie=504966108219016867,
table_id=10, datapath=datapath, priority=15,
match=match, instructions=instrs)
datapath.send_msg(flow_mod)
A10: Earlier, we have pointed out the experimenter match field TLV is incorrect as well.
Class: OFPXMC_EXPERIMENTER (0xffff) 0001 000. = Field: 8 .... ...0 = Has mask: False Length: 10 Experimenter: 0x00001018 Experimenter Value: 0008000000c8 => 000000c8
Please refer to the spec for the correct format.
Type => Class + Filed + MH Length => Length of Value Value => Experimenter + Experimenter Value
The RYU parser encodes the incorrect OFDPA experimenter “ofdpa_oxm_mpls_l2_port” and “ofdpa_oxm_mpls_type match” fields in SET_FIELD action.
MPLS L2 Port is type 8, therefore the Filed is 8.
MPLS L2 Port Experimenter value size is 32 bits.
The field type 8 is MPLS L2 Port and its Experimenter Value should be 000000c8, but your value is 0008000000c8.
The field type 23 is MPLS Type and its Experimenter Value should be 0001, but your value is 00170001.
RYU packet:
Action
Type: OFPAT_SET_FIELD (25)
Length: 24
OXM field
Class: OFPXMC_EXPERIMENTER (0xffff)
0001 000. = Field: 8 <---- OFDPA_OXM_MPLS_L2_Port (8)
.... ...0 = Has mask: False
Length: 10
Experimenter: 0x00001018
Experimenter Value: 0008000000c8 <---- your value is 0008000000c8 (MPLS L2 Port is 32 bits)
Unknown OXM body.
[Expert Info (Note/Undecoded): Unknown OXM body.]
Action
Type: OFPAT_SET_FIELD (25)
Length: 16
OXM field
Class: OFPXMC_EXPERIMENTER (0xffff)
0010 111. = Field: 23 <---- OFDPA_OXM_MPLS_TYPE (23)
.... ...0 = Has mask: False
Length: 8
Experimenter: 0x00001018
Experimenter Value: 00170001 <---- your value is 00170001 (MPLS_TYPE 16 bits)
Our packet:
Action
Type: OFPAT_SET_FIELD (25)
Length: 16
OXM field
Class: OFPXMC_EXPERIMENTER (0xffff)
0001 000. = Field: 8 <---- OFDPA_OXM_MPLS_L2_Port (8)
.... ...0 = Has mask: False
Length: 8
Experimenter: 0x00001018
Experimenter Value: 000000c8 <---- the value should be 000000c8 (MPLS L2 Port is 32 bits)
Unknown OXM body.
[Expert Info (Note/Undecoded): Unknown OXM body.]
And the Length should be 8 (4+4), not 10.
So the value should be 0x00001018000000c8, not 0x000010180008000000c8.
That means that the packet format which is generated by script is incorrect.
If you use the ovs-ofctl to issue the same flow, the flow can be installed successfully.
ovs-ofctl -O OpenFlow13 add-flow bridge_ofdpa table=10,vlan_vid=0x10c8,in_port=12,actions=set_field:0x100c8-\>tunnel_id,set_field:0xc8-\>ofdpa_mpls_l2_port,set_field:1-\>ofdpa_mpls_type,goto_table:13
Q11: [A5] We checked your MPLS L2 VPN group (type 9 subtype 1) in the Group Mod message (frame 13), there is no mpls ttl action in your Group Mod.
Please check the RYU about ofdpa experimenter.
In the script we provided:
# Add MPLS L2 VPN VPWS Group
l2vpn_group_id = self.generate_mpls_l2_vpn(cust_vrf)
actions = [ parser.OfdpaActionPushL2Header(), # Push L2 Header
parser.OFPActionPushVlan(0x8100),
parser.OFPActionPushMpls(0x8847),
parser.OFPActionSetField(mpls_label=mpls_srvlabel),
parser.OFPActionSetField(mpls_bos=1),
parser.OFPActionSetField(mpls_ttl=240),
parser.OfdpaActionPushCW(), # Push CW
parser.OFPActionGroup(b_mpls_tunnel_group)]
buckets = [parser.OFPBucket(0, 4294967295, 4294967295, actions)]
req = parser.OFPGroupMod(datapath, ofproto.OFPGC_ADD, ofproto.OFPGT_INDIRECT, l2vpn_group_id, buckets)
datapath.send_msg(req)
A11: The RYU parses “OFPActionSetField(mpls_ttl=240)” to encode the SET_FIELD action with incorrect NXM experimenter match field.
It should be encoded to the OFDPA experimenter OFDPA_OXM_MPLS_TTL match field.
RYU packet:
Action
Type: OFPAT_SET_FIELD (25)
Length: 16
OXM field
Class: OFPXMC_NXM_1 (0x0001) <---- NXM experimenter
0011 110. = Field: 30 <---- NXM MPLS_TTL (30)
.... ...0 = Has mask: False
Length: 1
Unknown OXM body.
Pad: 00000000000000
Our packet:
Action
Type: OFPAT_SET_FIELD (25)
Length: 16
OXM field
Class: OFPXMC_EXPERIMENTER (0xffff) <---- OFDPA experimenter
0000 111. = Field: 7 <---- OFDPA MPLS_TTL (7)
.... ...0 = Has mask: False
Length: 5
Experimenter: 0x00001018
Experimenter Value: f0
Unknown OXM body.
Pad: 0000
Q12: [A6] The table 25 is read-only, it can’t add flow, only used to go to this table.
The original spec uses tables 23 and 24, in your spec we see tables 24 and 25.
If table 25 is read-only, they how to work with two mpls tags?
I.e., we have an ingress mpls frame, we match it by its outer tag, then pop and send it to the next table to perform match by the inner tag (for stitching services). How to do it using tables 24 and 25?
A12: Please reference the Table 80 MPLS Flow Table 1 and 2 Flow Table Entry Types at OFDPA 3.0.4.0 specification page 98.
Table 24: E.g. L3 VPN Forward at OFDPA 3.0.4.0 specification page 100. ovs-ofctl -O OpenFlow13 add-flow bridge_ofdpa table=24,dl_type=0x8847,mpls_label=0x23456,mpls_bos=1,actions=set_field:1-\>ofdpa_vrf,set_field:32-\>ofdpa_mpls_type,write_actions\(group:0x20000001\),goto_table:27
Table 25: E.g. Pop Tunnel Label and Forward Based on Next Label at OFDPA 3.0.4.0 specification page 98. ovs-ofctl -O OpenFlow13 add-flow bridge_ofdpa table=24,dl_type=0x8847,mpls_label=0x23451,mpls_bos=0,actions=pop_mpls:0x8847,goto_table:25
Q13: [A10] We have pointed out the experimenter match field TLV is incorrect as well.
Referring to the OF-DPA Specification Version 3.03, Page 250:
The oxm_class field in the header is set to OFPXMC_EXPERIMENTER. Following the examples in the
ONF Extensions documents, the oxm field contains a value from ofdpa_match_exp_fields.
In the openflow-switch-extension-ext256.2.pdf document:
/* Structure for OXM field output match. */
struct onf_oxm_pbb_uca {
uint32_t oxm_header; /* oxm_class = OFPXMC_EXPERIMENTER,
oxm_field = ONFOXM_ET_PBB_UCA. */
uint32_t experimenter; /* ONF_EXPERIMENTER_ID. */
uint16_t reserved; /* Backward compatibility. */
uint8_t pbb_uca; /* PBB UCA header field. */
};
OFP_ASSERT(sizeof(struct onf_oxm_pbb_uca) == 11);
The reserved field should be set to zero.
This means that the Experimenter Value must be 0000000000c8 if we follow the standard.
The Length should be 10.
A13: The PBB UCA match field in openflow-switch-extension-ext256.2.pdf is only for ONF experimenter, not for OFDPA experimenter.
The OFDPA experimenter match field doesn’t have the reserved filed. Therefore the length should not be 10.
Please check the OFDPA match field definition on the 6.5.4 Match Fields chapter of OFDPA v3.0.4 specification.
These use the OpenFlow Experimenter OXM TLV definition as follows:
struct OFDPA_oxm_match_field {
uint32_t oxm_header; /* oxm_class = OFPXMC_ EXPERIMENTER */
uint32_t experimenter; /* Experimenter ID: 00-00-10-18 */
};
OFP_ASSERT(sizeof(struct OFDPA_oxm_match_field) == 8);
Q14: [A13] PBB UCA is an ONF Extension document and it is for the ONF experiment match field.
From the OFDPA 3.0 documentation, we have a clear reference that OFDPA follows ONF Extensions documents
“The oxm_class field in the header is set to OFPXMC_EXPERIMENTER. Following the examples in the
ONF Extensions documents, this is followed by a 2 byte …… ”
Moreover if you look at Copy_field BCM experimenter on 6.5.4, you can find that after the Experimenter_id follow 2-byte exp-type
(that confirm ONF Extention openflow-switch-extension-ext256.pdf)
struct onf_act_copy_field {
uint16_t type; /* OFPAT_EXPERIMENTER. */
uint16_t len; /* Length is padded to 64 bits. */
uint32_t experimenter; /* ONF_EXPERIMENTER_ID. */
uint16_t exp_type; /* ONFACT_ET_COPY_FIELD. */
uint8_t pad2[2];
uint16_t n_bits; /* Number of bits to copy. */
uint16_t src_offset; /* Starting bit offset in source. */
uint16_t dst_offset; /* Starting bit offset in destination. */
uint8_t pad[2]; /* Align to 32 bits. */
/* Followed by:
* - Exactly 8, 12 or 16 bytes containing the oxm_ids, then
* - Enough all-zero bytes (either 0 or 4) to make the action a whole
* multiple of 8 bytes in length */
uint32_t oxm_ids[0]; /* Source and destination OXM headers */
};
OFP_ASSERT(sizeof(struct ofp_action_copy_field) == 20);
A14: Please check the 6.5.4 Match Fields of OFDPA specification. The OFDPA specification doesn’t say “this is followed by a 2 byte ……”.
6.5.4 Match Fields
struct ofdpa_oxm_match_field {
uint32_t oxm_header; /* oxm_class = OFPXMC_ EXPERIMENTER */
uint32_t experimenter; /* Experimenter ID: 00-00-10-18 */
};
OFP_ASSERT(sizeof(struct ofdpa_oxm_match_field) == 8)
The oxm_class field in the header is set to OFPXMC_EXPERIMENTER. Following the examples in the
ONF Extensions documents, the oxm field contains a value from ofdpa_match_exp_fields. This is
followed by the experimenter id and any arguments as specified in Table 293.
Please check the 6.5.3 Actions of OFDPA specification. The Experimenter action has the exp_type, but the Experimenter match fields don’t have the exp_type.
6.5.3 Actions Actions that consists of an experimenter action header and a type code (exp_type) value from ofdpa_action_exp_type.
struct ofp_action_experimenter_header {
uint16_t type; /* OFPAT_EXPERIMENTER. */
uint16_t len; /* Length is a multiple of 8. */
uint32_t experimenter; /* Experimenter ID which takes the same
form as in struct
ofp_experimenter_header. */
};
OFP_ASSERT(sizeof(struct ofp_action_experimenter_header) == 8);
struct ofdpa_action_set_mpls_tc_from_vpn_table {
struct ofp_action_experimenter_header header; /* 8 bytes, OFDPA_EXPERIMENTER_ID */
uint16_t exp_type; /* OFDPA_ACT_SET_MPLS_TC_FROM_VPN_TABLE */
uint16_t qos_index; /* qos index argument */
uint8_t traffic_class; /* traffic class, supplied by switch */
uint8_t color; /* color, supplied by switch */
uint8_t pad[2]; /* align message on 64-bit boundary */
};
OFP_ASSERT(sizeof(struct ofdpa_action_set_mpls_tc_from_vpn_table) == 16);
