OVSDB system test
Luis Gomez <luis.gomez@...>
Hi folks,
Just started writing system test plan for OVSDB and I found one issue, one possible improvement and one general question.
1) Lets starts with the issue: with a controller with 2 IP interfaces (like the one we use for system test) connected to mininet OVS switch in passive mode (sudo ovs-vsctl set-manager ptcp:6640), when I add a bridge using:
POST http://10.125.136.52:8080/controller/nb/v2/networkconfig/bridgedomain/bridge/OVS/MINI1/s3
this one gets configured with the 2 IP addresses of the controller:
Bridge "s3" Controller "tcp:10.125.136.38:6633" Controller "tcp:10.125.136.52:6633" is_connected: true Port "s3" Interface "s3" type: internal
and this generates some instability in the controller (it adds and deletes the switch all the time):
2013-11-24 18:49:17.890 PST [ControllerI/O Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.53:33818 is connected to the Controller 2013-11-24 18:49:17.895 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Replacing existing Switch:10.125.136.39:52330 SWID:00:00:aa:e7:9a:2f:71:4b with New Switch:10.125.136.53:33818 SWID:00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:17.896 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.39:52330 SWID:00:00:aa:e7:9a:2f:71:4b is removed 2013-11-24 18:49:17.900 PST [FRM EventHandler Collector] INFO o.o.c.f.i.ForwardingRulesManager - Cleaning Flow database for Node OF|00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:18.890 PST [ControllerI/O Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.39:52332 is connected to the Controller 2013-11-24 18:49:18.895 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Replacing existing Switch:10.125.136.53:33818 SWID:00:00:aa:e7:9a:2f:71:4b with New Switch:10.125.136.39:52332 SWID:00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:18.898 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.53:33818 SWID:00:00:aa:e7:9a:2f:71:4b is removed 2013-11-24 18:49:18.900 PST [FRM EventHandler Collector] INFO o.o.c.f.i.ForwardingRulesManager - Cleaning Flow database for Node OF|00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:19.890 PST [ControllerI/O Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.53:33820 is connected to the Controller 2013-11-24 18:49:19.893 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Replacing existing Switch:10.125.136.39:52332 SWID:00:00:aa:e7:9a:2f:71:4b with New Switch:10.125.136.53:33820 SWID:00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:19.894 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.39:52332 SWID:00:00:aa:e7:9a:2f:71:4b is removed 2013-11-24 18:49:19.896 PST [FRM EventHandler Collector] INFO o.o.c.f.i.ForwardingRulesManager - Cleaning Flow database for Node OF|00:00:aa:e7:9a:2f:71:4b …..
And same behavior is observed when I start mininet with “sudo mn --controller=remote,ip=10.125.136.52 --topo linear,2” and then I make the connection to the OVS switch: both OF switches try to connect to both controller interfaces.
The solution would be to configure only the IP of the controller that is used (by linux routing) to reach the mininet VM? In this example 10.125.136.52.
Also this is not a strange scenario, I think the controller in a normal deployment will have at least 2 interfaces (operation and traffic) so it is good to fix this behavior.
2) The improvement is on the BridgeDomain NB API, if Ilook at the different methods available: I can add (POST) and remove (DELETE) bridge and ports configuration, but I do not see any method to list (GET) bridge/configuration. It would be nice to get a view of the bridge/port configuration from the controller NB.
3) And finally the question: The BridgeDomain POST method has a request body that it says it is documented under wiki.opendaylight.org project pages https://wiki.opendaylight.org/view/OVSDB_Integration:Mininet_OVSDB_Tutorial, however this page is not yet finished. Can you please share some examples of custom values for this field?
Thanks/Luis
|
Brent Salisbury
Hi Luis, Thanks so much for the feedback, this is really awesome! I threw in some thoughts, Im sure the others have additional input due to their better knowledge of the core controller but here are some starters. Since we automatically connect the OVS bridge instance to the controller we currently add all of the interfaces on the controller to the set-controller target until we decided the best way to tackle this. We just chatted about this in IRC (#opendaylight-ovsdb) over the weekend and we concluded a reasonable plan is to choose the interface on the controller containing the hosts default route and set the default target to that iface for each OVS bridge. It is on my list to code this evening so it will be patched shortly. The other would be a behavior if there was not a default route on the interface. It will likely be a random choice at that point since as you pointed out it gets very squirrely with multiples :) Please let us know if you have any other suggestions. - In the meantime we have recommended explicitly setting the interface to bind in the config.ini file. ### config.ini path/location ### distribution/opendaylight/target/distribution.opendaylight-osgipackage/opendaylight/configuration/config.ini ### config.ini file ### # Open Flow related system parameters # TCP port on which the controller is listening (default 6633) # of.listenPort=6633 # IP address of the controller (default: wild card) <== Uncomment and add the desired address for socket binding. # of.address = --Adjust that entry to an address you can bind to your host: ### config.ini file ### of.address = 172.16.58.1 <== I need to test this and see if it will accept a list also. 2. The following methods for example are wired from the IPluginInBridgeDomainConfigService. Modifying ports/bridges is still yet to be implemented. I was looking at it last night and help.append("---OVSDB CLI---\n"); help.append("\t ovsconnect <ConnectionName> <ip-address> - Connect to OVSDB\n"); help.append("\t addBridge <Node> <BridgeName> - Add Bridge\n"); help.append("\t getBridgeDomains <Node> - Get Bridges\n"); help.append("\t deleteBridgeDomain <Node> <BridgeName> - Delete a Bridge\n"); help.append("\t addPort <Node> <BridgeName> <PortName> <type> <options pairs> - Add Port\n"); help.append("\t deletePort <Node> <BridgeName> <PortName> - Delete Port\n"); help.append("\t addPortVlan <Node> <BridgeName> <PortName> <vlan> - Add Port, Vlan\n"); help.append("\t addTunnel <Node> <Bridge> <Port> <tunnel-type> <remote-ip> - Add Tunnel\n"); help.append(" printCache <Node> - On the configuration options we need to hash them out on how to match OVSDB specific or attributes not addressed by the ConfigConstants enum. Just putting them all into a CUSTOM object is sort of cantankerous. But I have not had a chance to get advice on this from the team. For example probably a bad idea but just pseudo code: bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getController()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getDatapath_id()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getExternal_ids()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getFail_mode()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getOther_config()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getPorts()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getStatus()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getStp_enable()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getDatapath_type()); 3. Thanks a bunch for pointing out the page. I just updated the Postman collection. I also linked a screen capture below to how it looks in the raw postman view. We will work on more documentation in coming days. BridgeDomain NBAPI An example screenshot of the Postman import: - Its ironic you mentioned the get ports for a given bridge. I coded list bridges for the "NodeConnector" method last night thinking it was just that. But then I asked Madhu and got clarity that the method is in fact for listing the connection between two nodes (per javadoc: Returns a NodeConnector mapped to a Port (if available) that is created using addPort). So maybe the best route is to just list them in the - I am also a bit confused on the method getBridgeDomainNode. The Javadoc for getBridgeDomainNode says "Returns a Node dedicated to a Bridge Domain (if available) that is created using createBridgeDomain.". I could probably think of a couple of things that could be but clarity on datapath, logical isolation and what not would help. - Please let me know if you have any time this week to review some more of this and it would be great if you had a little time to walk through some of your questions or potential issues you see with us. It is super helpful for us being heads down in code to get such awesome feedback from you! Regards, -Brent From: Luis Gomez <luis.gomez@...> Date: Sunday, November 24, 2013 10:40 PM To: "ovsdb-dev@..." <ovsdb-dev@...> Cc: "'integration-dev@...'" <integration-dev@...> Subject: [ovsdb-dev] OVSDB system test Hi folks,
Just started writing system test plan for OVSDB and I found one issue, one possible improvement and one general question.
1) Lets starts with the issue: with a controller with 2 IP interfaces (like the one we use for system test) connected to mininet OVS switch in passive mode (sudo ovs-vsctl set-manager ptcp:6640), when I add a bridge using:
POST http://10.125.136.52:8080/controller/nb/v2/networkconfig/bridgedomain/bridge/OVS/MINI1/s3
this one gets configured with the 2 IP addresses of the controller:
Bridge "s3" Controller "tcp:10.125.136.38:6633" Controller "tcp:10.125.136.52:6633" is_connected: true Port "s3" Interface "s3" type: internal
and this generates some instability in the controller (it adds and deletes the switch all the time):
2013-11-24 18:49:17.890 PST [ControllerI/O Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.53:33818 is connected to the Controller 2013-11-24 18:49:17.895 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Replacing existing Switch:10.125.136.39:52330 SWID:00:00:aa:e7:9a:2f:71:4b with New Switch:10.125.136.53:33818 SWID:00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:17.896 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.39:52330 SWID:00:00:aa:e7:9a:2f:71:4b is removed 2013-11-24 18:49:17.900 PST [FRM EventHandler Collector] INFO o.o.c.f.i.ForwardingRulesManager - Cleaning Flow database for Node OF|00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:18.890 PST [ControllerI/O Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.39:52332 is connected to the Controller 2013-11-24 18:49:18.895 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Replacing existing Switch:10.125.136.53:33818 SWID:00:00:aa:e7:9a:2f:71:4b with New Switch:10.125.136.39:52332 SWID:00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:18.898 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.53:33818 SWID:00:00:aa:e7:9a:2f:71:4b is removed 2013-11-24 18:49:18.900 PST [FRM EventHandler Collector] INFO o.o.c.f.i.ForwardingRulesManager - Cleaning Flow database for Node OF|00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:19.890 PST [ControllerI/O Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.53:33820 is connected to the Controller 2013-11-24 18:49:19.893 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Replacing existing Switch:10.125.136.39:52332 SWID:00:00:aa:e7:9a:2f:71:4b with New Switch:10.125.136.53:33820 SWID:00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:19.894 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.39:52332 SWID:00:00:aa:e7:9a:2f:71:4b is removed 2013-11-24 18:49:19.896 PST [FRM EventHandler Collector] INFO o.o.c.f.i.ForwardingRulesManager - Cleaning Flow database for Node OF|00:00:aa:e7:9a:2f:71:4b …..
And same behavior is observed when I start mininet with “sudo mn --controller=remote,ip=10.125.136.52 --topo linear,2” and then I make the connection to the OVS switch: both OF switches try to connect to both controller interfaces.
The solution would be to configure only the IP of the controller that is used (by linux routing) to reach the mininet VM? In this example 10.125.136.52.
Also this is not a strange scenario, I think the controller in a normal deployment will have at least 2 interfaces (operation and traffic) so it is good to fix this behavior.
2) The improvement is on the BridgeDomain NB API, if Ilook at the different methods available: I can add (POST) and remove (DELETE) bridge and ports configuration, but I do not see any method to list (GET) bridge/configuration. It would be nice to get a view of the bridge/port configuration from the controller NB.
3) And finally the question: The BridgeDomain POST method has a request body that it says it is documented under wiki.opendaylight.org project pages https://wiki.opendaylight.org/view/OVSDB_Integration:Mininet_OVSDB_Tutorial, however this page is not yet finished. Can you please share some examples of custom values for this field?
Thanks/Luis
|
Luis Gomez <luis.gomez@...>
Hi Brent,
Thanks for your quick answer. I will try the examples from Postman and let you know. For the question about which controller IP should be configured in the OVS bridge, I still think ideally it should be the IP address of the interface the controller uses to reach the OVS node, this is to be consistent with the idea of sending and receiving on the same interface. But maybe this is difficult to code and in that case the config.ini file seems a good workaround as well.
BR/Luis
From: Brent Salisbury [mailto:brent.salisbury@...]
Sent: Sunday, November 24, 2013 8:49 PM To: Luis Gomez; ovsdb-dev@... Cc: 'integration-dev@...' Subject: Re: [ovsdb-dev] OVSDB system test
Hi Luis,
Thanks so much for the feedback, this is really awesome! I threw in some thoughts, Im sure the others have additional input due to their better knowledge of the core controller but here are some starters.
Since we automatically connect the OVS bridge instance to the controller we currently add all of the interfaces on the controller to the set-controller target until we decided the best way to tackle this. We just chatted about this in IRC (#opendaylight-ovsdb) over the weekend and we concluded a reasonable plan is to choose the interface on the controller containing the hosts default route and set the default target to that iface for each OVS bridge. It is on my list to code this evening so it will be patched shortly. The other would be a behavior if there was not a default route on the interface. It will likely be a random choice at that point since as you pointed out it gets very squirrely with multiples :) Please let us know if you have any other suggestions.
- In the meantime we have recommended explicitly setting the interface to bind in the config.ini file.
### config.ini path/location ### distribution/opendaylight/target/distribution.opendaylight-osgipackage/opendaylight/configuration/config.ini ### config.ini file ### # Open Flow related system parameters # TCP port on which the controller is listening (default 6633) # of.listenPort=6633 # IP address of the controller (default: wild card) <== Uncomment and add the desired address for socket binding. # of.address =
--Adjust that entry to an address you can bind to your host: ### config.ini file ### of.address = 172.16.58.1 <== I need to test this and see if it will accept a list also.
2. The following methods for example are wired from the IPluginInBridgeDomainConfigService. Modifying ports/bridges is still yet to be implemented. I was looking at it last night and
help.append("---OVSDB CLI---\n"); help.append("\t ovsconnect <ConnectionName> <ip-address> - Connect to OVSDB\n"); help.append("\t addBridge <Node> <BridgeName> - Add Bridge\n"); help.append("\t getBridgeDomains <Node> - Get Bridges\n"); help.append("\t deleteBridgeDomain <Node> <BridgeName> - Delete a Bridge\n"); help.append("\t addPort <Node> <BridgeName> <PortName> <type> <options pairs> - Add Port\n"); help.append("\t deletePort <Node> <BridgeName> <PortName> - Delete Port\n"); help.append("\t addPortVlan <Node> <BridgeName> <PortName> <vlan> - Add Port, Vlan\n"); help.append("\t addTunnel <Node> <Bridge> <Port> <tunnel-type> <remote-ip> - Add Tunnel\n"); help.append(" printCache <Node>
- On the configuration options we need to hash them out on how to match OVSDB specific or attributes not addressed by the ConfigConstants enum. Just putting them all into a CUSTOM object is sort of cantankerous. But I have not had a chance to get advice on this from the team. For example probably a bad idea but just pseudo code:
bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getController()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getDatapath_id()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getExternal_ids()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getFail_mode()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getOther_config()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getPorts()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getStatus()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getStp_enable()); bridgeConfigs.put(ConfigConstants.CUSTOM, bridge.getDatapath_type());
3. Thanks a bunch for pointing out the page. I just updated the Postman collection. I also linked a screen capture below to how it looks in the raw postman view. We will work on more documentation in coming days.
BridgeDomain NBAPI
An example screenshot of the Postman import:
- Its ironic you mentioned the get ports for a given bridge. I coded list bridges for the "NodeConnector" method last night thinking it was just that. But then I asked Madhu and got clarity that the method is in fact for listing the connection between two nodes (per javadoc: Returns a NodeConnector mapped to a Port (if available) that is created using addPort). So maybe the best route is to just list them in the
- I am also a bit confused on the method getBridgeDomainNode. The Javadoc for getBridgeDomainNode says "Returns a Node dedicated to a Bridge Domain (if available) that is created using createBridgeDomain.". I could probably think of a couple of things that could be but clarity on datapath, logical isolation and what not would help.
- Please let me know if you have any time this week to review some more of this and it would be great if you had a little time to walk through some of your questions or potential issues you see with us. It is super helpful for us being heads down in code to get such awesome feedback from you!
Regards, -Brent
From: Luis Gomez <luis.gomez@...>
Hi folks,
Just started writing system test plan for OVSDB and I found one issue, one possible improvement and one general question.
1) Lets starts with the issue: with a controller with 2 IP interfaces (like the one we use for system test) connected to mininet OVS switch in passive mode (sudo ovs-vsctl set-manager ptcp:6640), when I add a bridge using:
POST http://10.125.136.52:8080/controller/nb/v2/networkconfig/bridgedomain/bridge/OVS/MINI1/s3
this one gets configured with the 2 IP addresses of the controller:
Bridge "s3" Controller "tcp:10.125.136.38:6633" Controller "tcp:10.125.136.52:6633" is_connected: true Port "s3" Interface "s3" type: internal
and this generates some instability in the controller (it adds and deletes the switch all the time):
2013-11-24 18:49:17.890 PST [ControllerI/O Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.53:33818 is connected to the Controller 2013-11-24 18:49:17.895 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Replacing existing Switch:10.125.136.39:52330 SWID:00:00:aa:e7:9a:2f:71:4b with New Switch:10.125.136.53:33818 SWID:00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:17.896 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.39:52330 SWID:00:00:aa:e7:9a:2f:71:4b is removed 2013-11-24 18:49:17.900 PST [FRM EventHandler Collector] INFO o.o.c.f.i.ForwardingRulesManager - Cleaning Flow database for Node OF|00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:18.890 PST [ControllerI/O Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.39:52332 is connected to the Controller 2013-11-24 18:49:18.895 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Replacing existing Switch:10.125.136.53:33818 SWID:00:00:aa:e7:9a:2f:71:4b with New Switch:10.125.136.39:52332 SWID:00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:18.898 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.53:33818 SWID:00:00:aa:e7:9a:2f:71:4b is removed 2013-11-24 18:49:18.900 PST [FRM EventHandler Collector] INFO o.o.c.f.i.ForwardingRulesManager - Cleaning Flow database for Node OF|00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:19.890 PST [ControllerI/O Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.53:33820 is connected to the Controller 2013-11-24 18:49:19.893 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Replacing existing Switch:10.125.136.39:52332 SWID:00:00:aa:e7:9a:2f:71:4b with New Switch:10.125.136.53:33820 SWID:00:00:aa:e7:9a:2f:71:4b 2013-11-24 18:49:19.894 PST [SwitchEvent Thread] INFO o.o.c.p.o.core.internal.Controller - Switch:10.125.136.39:52332 SWID:00:00:aa:e7:9a:2f:71:4b is removed 2013-11-24 18:49:19.896 PST [FRM EventHandler Collector] INFO o.o.c.f.i.ForwardingRulesManager - Cleaning Flow database for Node OF|00:00:aa:e7:9a:2f:71:4b …..
And same behavior is observed when I start mininet with “sudo mn --controller=remote,ip=10.125.136.52 --topo linear,2” and then I make the connection to the OVS switch: both OF switches try to connect to both controller interfaces.
The solution would be to configure only the IP of the controller that is used (by linux routing) to reach the mininet VM? In this example 10.125.136.52.
Also this is not a strange scenario, I think the controller in a normal deployment will have at least 2 interfaces (operation and traffic) so it is good to fix this behavior.
2) The improvement is on the BridgeDomain NB API, if Ilook at the different methods available: I can add (POST) and remove (DELETE) bridge and ports configuration, but I do not see any method to list (GET) bridge/configuration. It would be nice to get a view of the bridge/port configuration from the controller NB.
3) And finally the question: The BridgeDomain POST method has a request body that it says it is documented under wiki.opendaylight.org project pages https://wiki.opendaylight.org/view/OVSDB_Integration:Mininet_OVSDB_Tutorial, however this page is not yet finished. Can you please share some examples of custom values for this field?
Thanks/Luis
_______________________________________________ ovsdb-dev mailing list ovsdb-dev@... https://lists.opendaylight.org/mailman/listinfo/ovsdb-dev |