[sfc-dev] [opnfv-tech-discuss] [SFC] Slides to show OpenStack NSH Issues

Daniel Smith daniel.smith at ericsson.com
Mon Aug 31 15:59:36 UTC 2015

Hello there.

In the SDN tunneling / OVSDB integration Demo we showed, we have seen that having 2 VM’s sitting on the same Neutron created VXLAN network can talk to each other fine.  However, as has been indicated the issue is going to be where you want (in the OVS stack) to do the NSH header insertion.  This could be done using an overload of the NIC driver (a DPDK poller method that has OVS sitting on top untouched) or you can do it in OVS (but then have to content with the rules separating traffic flows in OVS).

We know that we can get two VM’s to talk to one another over a VXLAN tunnel each – so long as they are in the same neutron network (and in the same tenant) if they are to be in different subnets (the VMs(Sf’s)) and thus different neutron vxlan nets, then yes – you will need to route or do some magic at the physical layer if you want them to talk to one another.


From: opnfv-tech-discuss-bounces at lists.opnfv.org [mailto:opnfv-tech-discuss-bounces at lists.opnfv.org] On Behalf Of Sam Hague
Sent: Monday, August 31, 2015 11:56 AM
To: Brady Allen Johnson
Cc: opnfv-tech-discuss at lists.opnfv.org; sfc-dev at lists.opendaylight.org; Edward Warnicke
Subject: Re: [opnfv-tech-discuss] [sfc-dev] [SFC] Slides to show OpenStack NSH Issues

that was one of the proposed solutions- but as you mention, it needs to be tested. Setting the port type to VxLAN is not hard to do - whether from OpenStack or via ovsdb.  The real issue, is will using a VxLAN port as the VM port actually work? There is a big difference between forwarding packets via a tap port and via a VxLAN port. That really needs to be tested to verify if it will work.

On Mon, Aug 31, 2015 at 11:46 AM, Brady Allen Johnson <brady.allen.johnson at ericsson.com<mailto:brady.allen.johnson at ericsson.com>> wrote:
Hello everybody,

Im back from vacation today, and have been getting caught-up on the different email threads.

Wouldnt a simpler solution to all of this be to get OpenStack to start the VM with a VXLAN port? If we could configure OpenStack to do this, then as I understand it, we've solved this entire problem, right?

I was chatting with Dan Smith about this, and he says it is indeed possible to configure Neutron to start a VM with a VXLAN port. Does anybody have time to test this?



On 08/31/2015 08:33 AM, Zhou, Danny wrote:
Echo Ed on the statement “The root problem is that when you use a port to encap, you have committed to having the local IP stack deliver it, and loose control over the OUTPUT port for the encapped packet.” with more details below:

1)      In current ODL SFC code, specifically in SFCOFL2, in order for SFF to steer received VxLAN-GPE-NSH tunneling packets to a SF, it changes the outer L3 header’ destination IP from br-int’ to SF’ when VxLAN-GPE-NSH VTEP performs “decapsulation followed-by re-encapsulation” actions.  The vSwitch learns the OUTPUT port that connects to the destination SF from the ARP request/response session on the local IP stack. In other words, there is no direct traffic steering rules generated from the SFCOFL2 to forward decapsulated VxLAN-GPE-NSH tunneling packets to a SF via the output port. The SFCOFL2 should be changed to accommodate this, assuming once Tacker creates a VNF per request from SFC it should return the Neutron port info (except for VNF’ IP) to SFC as well so SFC could leverage it. But the problem is TAP port does not allow pass VxLAN/NSH headers to the VNF.

2)      To enable output port control for TAP device, we might have to either change kernel VTEP to enable pass VxLAN/NSH packets as regular L2 frame or enable the L2/NSH in kernel so packet can be forwarded to TAP directly. BTW, for DPDK-netdev OVS’ NSH patch, the DPDK-netdev uses user space vHost as backend to hook into virtio PV driver in a VM, so no TAP device is used hence packets with VxLAN/NSH headers can be forwarded to SF directly via specifying output port in the forwarding rule’ “action”.

From: sfc-dev-bounces at lists.opendaylight.org<mailto:sfc-dev-bounces at lists.opendaylight.org> [mailto:sfc-dev-bounces at lists.opendaylight.org] On Behalf Of Edward Warnicke
Sent: Saturday, August 29, 2015 3:21 AM
To: Sam Hague
Cc: sfc-dev at lists.opendaylight.org<mailto:sfc-dev at lists.opendaylight.org>; opnfv-tech-discuss at lists.opnfv.org<mailto:opnfv-tech-discuss at lists.opnfv.org>
Subject: Re: [sfc-dev] [SFC] Slides to show OpenStack NSH Issues


I had a long talk on IRC with Tim about this, and I believe we have misidentified the root issue.

The problem is not that the VXLAN (or other) encap is being stripped off by the bridge.  The proper tunnel dest for the frame being delivered to the SF
 is the IP of the SF, not the bridge. The problem is that we can only encap by sending the packet to a tunnel port on OVS, and
that tunnel port then dumps to the hosts local IP stack.  It should be noted, that this is an issue for tunnels in general for OVS, not just VXLAN, or NSH.

The root problem is that when you use a port to encap, you have committed to having the local IP stack deliver it, and loose control over the OUTPUT
port for the encapped packet.

Looking at the proposed solutions, the "Hybrid VxLAN Tunnel" solution proposed doesn't work because it relies on us manipulating the local hosts
IP routing tables, and if you get overlapping IP spaces (which you will) that will fail (among other issues).

This same issue is going to break the other proposed solutions:

"Change Port Type from tap to VxLAN", "NSH Proxy", and "Add Linux Bridge"

If we had a proper push/pop vxlan header OpenFlow Action, we could simply use that to encap and output to the port of our choosing, including
the tap port.

I think this is what you mean with the solution "Modify OVS to pass VxLAN/NSH over Tap Ports".  As far as I can see, this is really the only
viable option.

I would suggest we look into getting such a proper solution to the root issue, rather than hacking around in ways that are likely to produce even
more technical debt.


On Thu, Aug 27, 2015 at 10:32 AM, Sam Hague <shague at redhat.com<mailto:shague at redhat.com>> wrote:

they do - but at that point the VxLAN/NSH has already been stripped on ingress to the bridge. The headers are stored in metadata but that doesn't get added back unless the packets egresses a vlxan port.

So some of the solutions revolve around sending the packets out a VxLAN port so the headers are added and try to get that packet into the SF - without the headers stripped or packets dropped. But you need to bypass the OVS desire to expect vtep ports when it received VxLAN encapped packets. We are in the middle of testing variations on how to do that.

Tim mentioned you had a variation of using the IP of the SF to do that bypass mentioned above to get the encapped packets sneaking through the bridge to the SF. That is on the list of things to try since it is a slight variation.


On Thu, Aug 27, 2015 at 12:13 PM, Edward Warnicke <hagbard at gmail.com<mailto:hagbard at gmail.com>> wrote:

Question... don't tap ports simply carry ethernet frames up to the guest?


On Thu, Aug 27, 2015 at 8:57 AM, Sam Hague <shague at redhat.com<mailto:shague at redhat.com>> wrote:
Hi all,

I had the acttion items to create slides showing the issues when using OpenStack instantiated VMs that cannot terminate VxLAN/NSH tunnels. Those are at [slides]. W can use this as a basis for working out the solutions.

Thanks, Sam

[slides] https://docs.google.com/presentation/d/1y0A1DNA1bB6y-1hR-ZfvtW8cKL4psMvj8BdaEMC4n2I/edit?usp=sharing

sfc-dev mailing list
sfc-dev at lists.opendaylight.org<mailto:sfc-dev at lists.opendaylight.org>


sfc-dev mailing list

sfc-dev at lists.opendaylight.org<mailto:sfc-dev at lists.opendaylight.org>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendaylight.org/pipermail/sfc-dev/attachments/20150831/9a371c29/attachment-0001.html>

More information about the sfc-dev mailing list