[sfc-dev] Service Chain Last Hop

Brady Allen Johnson brady.allen.johnson at ericsson.com
Mon Aug 31 13:18:36 UTC 2015


Keith,

We have this setup in a lab here, and can test with NORMAL. I'll let you 
know how it goes.

A suspicion that we have is that the packet still has the IPv4 Tun Dst 
IP = SFF, and OVS sees that and just drops it before getting to the 
point of decapsulating it.

Regards,

Brady


On 08/31/2015 03:07 PM, Keith Burns wrote:
>
> Brady,
>
> For the non-overlay egress case (ie referred to as nonGBP case) we 
> can't use INPORT as that is VXLAN.
>
> My patch tried changing this to NORMAL but since I couldn't get the 
> client going, couldn't test.
>
> If normal doesn't work, then we will need to plumb a interface into 
> OVS. Could be a host eth, could be a veth into a netns with access to 
> the correct routing etc.
>
> Simplest is to test normal.
>
> In fact anyone can test this without a patch. If you have something 
> that needs to see regular ipv4/ paks on egress, hand craft the same 
> flow but instead of flow actions INPORT do it w NORMAL bit w higher 
> priority ..
>
> On Aug 31, 2015 2:55 AM, "Brady Allen Johnson" 
> <brady.allen.johnson at ericsson.com 
> <mailto:brady.allen.johnson at ericsson.com>> wrote:
>
>
>     Hello,
>
>     I wanted to add my 2 cents to the discussion since I did the NSH
>     OpenFlow programming together with Keith, Reinaldo, and Paul. I
>     noticed some responses regarding the flows that werent completely
>     accurate, and have detailed exactly what each flow does below:
>
>     Here are the flows that Ricardo pasted previously. Notice that I
>     parsed them a bit with /*awk*/ to remove the cookie and other
>     superfluous fields.
>
>         ubuntu at sff:~$ sudo ovs-ofctl -O OpenFlow13 dump-flows ovs-br2
>         | awk '{print $3, $4, $6, $7}'
>
>         OFPST_FLOW reply (OF1.3) (xid=0x2):
>
>         table=0, n_packets=42, priority=250,ip actions=goto_table:3
>
>         table=0, n_packets=0, priority=5 actions=drop
>
>         table=1, n_packets=0, priority=5 actions=goto_table:2
>
>         table=2, n_packets=0, priority=5 actions=goto_table:3
>
>         table=3, n_packets=20, priority=5 actions=goto_table:10
>
>         table=3, n_packets=22, priority=540,nsp=1,nsi=255
>         actions=load:0xc0a8010c->NXM_NX_TUN_IPV4_DST[],goto_table:10
>
>         table=10, n_packets=0, priority=5 actions=drop
>
>         table=10, n_packets=22, priority=640,nsp=1,nsi=255
>         actions=move:NXM_NX_NSH_C1[]->NXM_NX_NSH_C1[],move:NXM_NX_NSH_C2[]->NXM_NX_NSH_C2[],move:NXM_NX_TUN_ID[0..31]->NXM_NX_TUN_ID[0..31],IN_PORT
>
>         table=10, n_packets=0, priority=640,nsp=1,nsi=254
>         actions=move:NXM_NX_NSI[]->NXM_NX_NSI[],move:NXM_NX_NSP[]->NXM_NX_NSP[],move:NXM_NX_NSH_C1[]->NXM_NX_TUN_IPV4_DST[],move:NXM_NX_NSH_C2[]->NXM_NX_TUN_ID[0..31],move:NXM_NX_TUN_ID[0..31]->NXM_NX_TUN_ID[0..31],IN_PORT
>
>         table=10, n_packets=20, priority=660,nsp=1,nsi=254,nshc1=0
>         actions=IN_PORT
>
>     The really important flows for NSH are the following (tables 1 and
>     2 are for non-NSH):
>
>         table=0, n_packets=42, priority=250,ip actions=goto_table:3
>             Table 0 is the "Transport Ingress" table, and in this case
>         checks for IP, and forwards the packets to table 3. In the
>         future, we may change this to check that the in_port is the
>         expected port.
>
>         table=3, n_packets=20, priority=5 actions=goto_table:10
>         →PACKETS COMING FROM THE SF
>
>         table=3, n_packets=22, priority=540,nsp=1,nsi=255
>         actions=load:0xc0a8010c->NXM_NX_TUN_IPV4_DST[],goto_table:10
>         →PACKETS GOING TO THE SF
>
>             Table 3 is the "Next Hop" table, and sets the next SF or
>         SFF accordingly.
>
>         table=10, n_packets=22, priority=640,nsp=1,nsi=255
>         actions=move:NXM_NX_NSH_C1[]->NXM_NX_NSH_C1[],move:NXM_NX_NSH_C2[]->NXM_NX_NSH_C2[],move:NXM_NX_TUN_ID[0..31]->NXM_NX_TUN_ID[0..31],IN_PORT
>
>         table=10, n_packets=0, priority=640,nsp=1,nsi=254
>         actions=move:NXM_NX_NSI[]->NXM_NX_NSI[],move:NXM_NX_NSP[]->NXM_NX_NSP[],move:NXM_NX_NSH_C1[]->NXM_NX_TUN_IPV4_DST[],move:NXM_NX_NSH_C2[]->NXM_NX_TUN_ID[0..31],move:NXM_NX_TUN_ID[0..31]->NXM_NX_TUN_ID[0..31],IN_PORT
>
>         table=10, n_packets=20, priority=660,nsp=1,nsi=254,nshc1=0
>         actions=IN_PORT
>
>             Table 10 is the "Transport Egress" and prepares the
>         packets for Egress. I'll explain each of these flows below.
>
>
>     Transport Egress flows detailed:
>
>     The first flow in table 10 is used to send the packet from the SFF
>     to the SF
>
>     table=10, n_packets=22, priority=640,nsp=1,nsi=255
>     actions=move:NXM_NX_NSH_C1[]->NXM_NX_NSH_C1[],move:NXM_NX_NSH_C2[]->NXM_NX_NSH_C2[],move:NXM_NX_TUN_ID[0..31]->NXM_NX_TUN_ID[0..31],IN_PORT
>
>
>     The actions do the following:
>
>       * copy the NSH context headers C1 and C2 from the ingress to the
>         egress
>       * copy the VxLAN tunnel Id from the ingress to the egress
>       * Send the packet back to the port it was received on. Since the
>         SFF only has 1 port, we dont need to specify the exact port.
>
>
>     The second and third flows are the Service Chain egress flows, as
>     described here:
>
>     table=10, n_packets=0, priority=640,nsp=1,nsi=254
>     actions=move:NXM_NX_NSI[]->NXM_NX_NSI[],move:NXM_NX_NSP[]->NXM_NX_NSP[],move:NXM_NX_NSH_C1[]->NXM_NX_TUN_IPV4_DST[],move:NXM_NX_NSH_C2[]->NXM_NX_TUN_ID[0..31],move:NXM_NX_TUN_ID[0..31]->NXM_NX_TUN_ID[0..31],IN_PORT
>
>
>     This flow is for the GBP+SFC integration
>
>     table=10, n_packets=20, priority=660,nsp=1,nsi=254,nshc1=0
>     actions=IN_PORT
>
>     This flow is for the Non-GBP case. "nshc1=0" is checking that the
>     NSH context header C1 is not present. This flow just sends the
>     packet back to the port it was received on. This flow seems to be
>     where the problem is when terminating the Service Chain. We tried
>     changing the action to "Normal" to send it to the kernel, but that
>     didnt work. The original idea was that sending it to "IN_PORT"
>     would decapsulate the VXLAN/NSH allowing the packet to be routed
>     to the original destination. But this raises a question: How does
>     the SFF OVS port know when to decapsulate the packet? That is,
>     when the SFF sends a packet to the SF, the OVS port should not
>     decapsulate, but when the service chain is terminated, we want the
>     OVS port to decapsulate. How does the port know when and when not
>     to decapsulate?
>
>     Regards,
>
>     Brady
>
>
>
>     On 08/28/2015 08:21 AM, Reinaldo Penno wrote:
>>     I tried that based on a patch from Brady but it did not work.
>>      But there is always a possibility that something was wrong.
>>
>>     From: "Xiao Liang (xiaolia)" <xiaolia at cisco.com
>>     <mailto:xiaolia at cisco.com>>
>>     Date: Thursday, August 27, 2015 at 10:33 PM
>>     To: "Zhou, Danny" <danny.zhou at intel.com
>>     <mailto:danny.zhou at intel.com>>, Reinaldo Penno <rapenno at gmail.com
>>     <mailto:rapenno at gmail.com>>
>>     Cc: "sfc-dev at lists.opendaylight.org
>>     <mailto:sfc-dev at lists.opendaylight.org>"
>>     <sfc-dev at lists.opendaylight.org
>>     <mailto:sfc-dev at lists.opendaylight.org>>
>>     Subject: RE: [sfc-dev] Service Chain Last Hop
>>
>>     Hi,
>>
>>     At the last hop SFF, can we just direct the original L3 packet
>>     (whose outer header has been removed by VTEP) to LOCAL port?
>>
>>     Thanks
>>
>>     *From:*sfc-dev-bounces at lists.opendaylight.org
>>     <mailto:sfc-dev-bounces at lists.opendaylight.org>
>>     [mailto:sfc-dev-bounces at lists.opendaylight.org] *On Behalf Of
>>     *Zhou, Danny
>>     *Sent:* Friday, August 28, 2015 12:12
>>     *To:* Reinaldo Penno
>>     *Cc:* sfc-dev at lists.opendaylight.org
>>     <mailto:sfc-dev at lists.opendaylight.org>
>>     *Subject:* Re: [sfc-dev] Service Chain Last Hop
>>
>>     Hi Reinaldo, I added inline comments with [DZ]
>>
>>     *From:*Reinaldo Penno [mailto:rapenno at gmail.com]
>>     *Sent:* Thursday, August 27, 2015 11:44 PM
>>     *To:* Zhou, Danny
>>     *Cc:* sfc-dev at lists.opendaylight.org
>>     <mailto:sfc-dev at lists.opendaylight.org>
>>     *Subject:* Re: [sfc-dev] Service Chain Last Hop
>>
>>     HI Danny,
>>
>>     I still think there is a disconnect about what is going on.  Let
>>     me try a different way.
>>
>>     Without SFC you have a client A and Server B that communicate
>>     with each other.
>>
>>     Client A (10.0.0.2) --------> Server B (10.0.0.100)
>>
>>     Now you put a SFC in the middle
>>
>>     SF (Firewall)
>>
>>          |
>>
>>     Client A ---> classifier ---> SFF ---> Server B
>>
>>                         |--- NSH —|
>>
>>     - After SFF gets a packet from the Firewall it checks that it is
>>     the end of the path and does the following:
>>
>>     1 – Removes NSH and any other outer encapsulation(VXLAN,
>>     Ethernet, etc).
>>
>>          At this point SFF has the original IP packet that A sent to B
>>
>>     2 – SFF then releases this original packet to the kernel
>>
>>     3 – kernel forwards this original packet to Server B as it would
>>     any other packet.
>>
>>     Premise:
>>
>>     - SPID is 1, SI is 255, which means SI = 254 when SFF last
>>     receives it.
>>
>>     - Encapsulation is VXLAN-GPE
>>
>>     Questions:
>>
>>     Can this be achieved with a single physical ethernet interface? 
>>     If more than one is needed than it would tremendously complicate
>>     unit testing.
>>
>>     What would be the OF rules look like?
>>
>>     [DZ]: No need to add an additional physical Ethernet interface. 
>>     Just forward the original packet out via eth1 (in the diagram
>>     below) device on the SFF no matter the eth1 is driven by kernel
>>     driver or DPDK PMD in user space, assuming the network middle
>>     boxes (e.g. router or switches) have been configured correctly to
>>     forward the original packet to the Server B.  The tricky on the
>>     VxLAN-GPE-NSH port on br-eth1 bridge of the SFF is it needs rule to
>>
>>     1)re-encapsulate the decapsulated tunneling packets if they come
>>     from physical port eth1,  and then forward the re-encapsulated
>>     packets with NSH header to SF (e.g. Firewall).
>>
>>     2)do not re-encapsulate the tunneling packets if they come from
>>     vport connecting to the SF and NSI = 0, so the original packets
>>     can be forwarded to the eth1 directly.
>>
>>     Please noted, the behavior of VTEP port on the bridge is that it
>>     automatically decapsulates the turning packets on the bridge.
>>
>>     I’m assuming we can not put individual rules for each possible
>>     client (there might be millions), so we need some sort of wildcard
>>
>>     Thanks,
>>
>>     *From: *"Zhou, Danny" <danny.zhou at intel.com
>>     <mailto:danny.zhou at intel.com>>
>>     *Date: *Thursday, August 27, 2015 at 6:07 AM
>>     *To: *"Zhou, Danny" <danny.zhou at intel.com
>>     <mailto:danny.zhou at intel.com>>, Reinaldo Penno <rapenno at gmail.com
>>     <mailto:rapenno at gmail.com>>
>>     *Cc: *"sfc-dev at lists.opendaylight.org
>>     <mailto:sfc-dev at lists.opendaylight.org>"
>>     <sfc-dev at lists.opendaylight.org
>>     <mailto:sfc-dev at lists.opendaylight.org>>
>>     *Subject: *RE: [sfc-dev] Service Chain Last Hop
>>
>>     For a typical setup in diagram below where SFF(NSH-aware OvS)
>>     connects to Classifier(NSH-aware OvS) via VxLAN-GPE-NSH tunnel,
>>     the OVSDB commandline for the Classifier to setup VTEP is:
>>
>>     # ovs-vsctl add-port br-int vxlan0 -- set interface vxlan0
>>     type=vxlan options:remote_ip=192.168.1.2 *options:dst_port=4790
>>     options:out_nsi=flow options:out_nsp=flow options:in_nsi=flow
>>     options:in_nsp=flow *
>>
>>     /* Note: UDP destination port set to 4790 means this is a
>>     VxLAN-GPE-NSH VTEP  rather than a regular VxLAN VTEP */
>>
>>     **
>>
>>     And the flow commandline is
>>
>>     # ovs-ofctl add-flow br-int “in_port=2, nw_src =10.0.0.3
>>     actions=output:1“
>>
>>     /* Note: Port1 on br-int connects to eth2 and Port2 connects to
>>     vxlan0 which is a VxLAN-GPE-NSH port. So when the classifier
>>     receives a tunneling packet from the SFF, the vxlan0 VTEP strips
>>     off VxLAN-GPE and NSH header then transmit a plain packet out via
>>     eth2 if its source IP is 10.0.0.3 */
>>
>>     SFC should generate equivalent Openflow messages via sfcofl2 and
>>     use OVSDB SB plugin to creates VTEP port on both classifier and SFF.
>>
>>     *From:*sfc-dev-bounces at lists.opendaylight.org
>>     <mailto:sfc-dev-bounces at lists.opendaylight.org>
>>     [mailto:sfc-dev-bounces at lists.opendaylight.org] *On Behalf Of
>>     *Zhou, Danny
>>     *Sent:* Thursday, August 27, 2015 12:17 PM
>>     *To:* Reinaldo Penno
>>     *Cc:* sfc-dev at lists.opendaylight.org
>>     <mailto:sfc-dev at lists.opendaylight.org>
>>     *Subject:* Re: [sfc-dev] Service Chain Last Hop
>>
>>     Unlike encapsulation which needs sfcofl2 to control VTEP what to
>>     fill the L2/L3 as well as NSH headers, decapsulation is default
>>     behavior of OVS VTEP which is controlled by OVSDB only, not by
>>     Openflow/sfcofl2. The NSH-aware VTEP is smart enough to strip
>>     VxLAN-GPE-NSH header if it detects it is VxLAN-GPE header,
>>     otherwise it removes VxLAN header only. Let me dig into it and
>>     come back with detailed solution.
>>
>>     *From:*Reinaldo Penno [mailto:rapenno at gmail.com]
>>     *Sent:* Thursday, August 27, 2015 12:06 PM
>>     *To:* Zhou, Danny
>>     *Cc:* Ricardo Noriega De Soto;
>>     <mailto:sfc-dev at lists.opendaylight.org>sfc-dev at lists.opendaylight.org
>>     <mailto:sfc-dev at lists.opendaylight.org>
>>     *Subject:* Re: [sfc-dev] Service Chain Last Hop
>>
>>     Right but the OF rules installed by sfcofl2 do not have this
>>     capability. Keith mentioned a patch but modulus testing and
>>     merging this possible patch the OVS solution has this limitation.
>>     Do you have idea on how to approach this problem?
>>
>>
>>     On Aug 26, 2015, at 8:38 PM, Zhou, Danny <danny.zhou at intel.com
>>     <mailto:danny.zhou at intel.com>> wrote:
>>
>>         Thanks Reinaldo. For that case, technically the NSH-aware OVS
>>         can be configured to remove the VxLAN-GPE-NSH header and send
>>         original L2 frame out, no matter the OvS acts as classifier
>>         or SFF.
>>
>>         *From:*Reinaldo Penno [mailto:rapenno at gmail.com]
>>         *Sent:* Thursday, August 27, 2015 11:32 AM
>>         *To:* Zhou, Danny
>>         *Cc:* Ricardo Noriega De Soto; sfc-dev at lists.opendaylight.org
>>         <mailto:sfc-dev at lists.opendaylight.org>
>>         *Subject:* Re: [sfc-dev] Service Chain Last Hop
>>
>>         Danny,
>>
>>         I think the question is about the last SFF, no more SF or
>>         anything else. Currently it can not decap VXLAN packet and
>>         send a plain IP packet out. Therefore we make the OVS send
>>         the packet an 'extra hop' that can actually decap the packet.
>>
>>         Thanks
>>
>>
>>         On Aug 26, 2015, at 8:23 PM, Zhou, Danny
>>         <danny.zhou at intel.com <mailto:danny.zhou at intel.com>> wrote:
>>
>>             NSH-aware OvS DOES decapsulate VxLAN-GPE-NSH tunneling
>>             packets. The trick part is that once decapsulation is
>>             done the VxLAN-GPE header is removed, in order for SFF to
>>             steer traffics to NSH-aware SF with NSH header, you have
>>             to add a transport before the NSH header otherwise the
>>             MAC layer in SF will drop them due to no L2 header. So
>>             OvS could 1) re-encapsulate the NSH + Original L2 frame
>>             with the VxLAN-GPE header (which is how patched OvS works
>>             today) or 2) add a Ethernet header before the NSH +
>>             Original L2 frame.
>>
>>             *From:*sfc-dev-bounces at lists.opendaylight.org
>>             <mailto:sfc-dev-bounces at lists.opendaylight.org>
>>             [mailto:sfc-dev-bounces at lists.opendaylight.org] *On
>>             Behalf Of *Ricardo Noriega De Soto
>>             *Sent:* Thursday, August 27, 2015 6:03 AM
>>             *To:* Reinaldo Penno
>>             *Cc:* sfc-dev at lists.opendaylight.org
>>             <mailto:sfc-dev at lists.opendaylight.org>
>>             *Subject:* Re: [sfc-dev] Service Chain Last Hop
>>
>>             Please, I’d appreciate if you answer these questions:
>>
>>             - Is the patched OVS NSH-aware capable of decapsulating
>>             traffic?
>>
>>             - Regarding this:
>>
>>                 ctx1 is the dstset in this header for the final hop?
>>
>>                 [RP] Correct. If you set this IPaddress to the
>>                 machine where the sff_client is running it will get
>>                 the packet back and print it out.
>>
>>             What if we want to get rid of the sff_client? Which would
>>             be the best way of doing that?
>>
>>             I see the sff_client as a testing tool, as Reinaldo
>>             mentioned, but nothing more than that. It is very
>>             important to use real traffic and make an infrastructure
>>             to encap/decap it transparently.
>>
>>                 El 26/8/2015, a las 20:38, Reinaldo Penno
>>                 <rapenno at gmail.com <mailto:rapenno at gmail.com>> escribió:
>>
>>                 For the benefit of the list since lots of folks
>>                 interested.
>>
>>                 *From: *Reinaldo Penno <rapenno at gmail.com
>>                 <mailto:rapenno at gmail.com>>
>>                 *Date: *Wednesday, August 26, 2015 at 11:36 AM
>>                 *To: *João Silva <silvajp9 at gmail.com
>>                 <mailto:silvajp9 at gmail.com>>
>>                 *Subject: *Re: [sfc-dev] Service Chain Last Hop
>>
>>                 *From: *João Silva <silvajp9 at gmail.com
>>                 <mailto:silvajp9 at gmail.com>>
>>                 *Date: *Wednesday, August 26, 2015 at 11:32 AM
>>                 *To: *Reinaldo Penno <rapenno at gmail.com
>>                 <mailto:rapenno at gmail.com>>
>>                 *Subject: *Re: [sfc-dev] Service Chain Last Hop
>>
>>                 Please, clear something for me: the outer header is
>>                 the the VxLANGPE right?
>>
>>                 [RP] Yes.
>>
>>                 ctx1 is the dstset in this header for the final hop?
>>
>>                 [RP] Correct. If you set this IPaddress to the
>>                 machine where the sff_client is running it will get
>>                 the packet back and print it out.
>>
>>                 On Wed, Aug 26, 2015 at 7:25 PM, Reinaldo Penno
>>                 <rapenno at gmail.com <mailto:rapenno at gmail.com>> wrote:
>>
>>
>>                 No, the context header determines the the destination
>>                 that will be used in the outer IP header.  It is
>>                 something to overcome the inability of OVS to decap
>>                 and send a plain IP packet out.
>>
>>                 *From: *João Silva <silvajp9 at gmail.com
>>                 <mailto:silvajp9 at gmail.com>>
>>                 *Date: *Wednesday, August 26, 2015 at 11:16 AM
>>                 *To: *Reinaldo Penno <rapenno at gmail.com
>>                 <mailto:rapenno at gmail.com>>
>>
>>
>>                 *Subject: *Re: [sfc-dev] Service Chain Last Hop
>>
>>                 That works fine. But what is sent to --ctx1 is the
>>                 encapsulated packet, is it not?
>>
>>                 On Wed, Aug 26, 2015 at 7:01 PM, Reinaldo Penno
>>                 <rapenno at gmail.com <mailto:rapenno at gmail.com>> wrote:
>>
>>
>>                 Please use the NSH client (sff_client) and send
>>                 packets with context headers filled.
>>
>>                 In general the way I test these things is the
>>                 following: I test with the sff_client and agent and
>>                 if it works I start complicating and/or substituting
>>                 pieces.
>>
>>                 In SFC 102 I show how you send a packet with context
>>                 header filled. Noticed the "--ctx1 192.168.1.7"
>>
>>                 "REPENNO-M-L9FE:sfc
>>
>
>     _______________________________________________
>     sfc-dev mailing list
>     sfc-dev at lists.opendaylight.org <mailto:sfc-dev at lists.opendaylight.org>
>     https://lists.opendaylight.org/mailman/listinfo/sfc-dev
>
>     ...
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendaylight.org/pipermail/sfc-dev/attachments/20150831/bd26ebfb/attachment-0001.html>


More information about the sfc-dev mailing list