z.archive.ovsdb-dev@lists.opendaylight.org | [integration-dev] [release] [genius-dev] Genius CSIT failing on master

Home Messages Hashtags Subgroups

This group is locked. No changes can be made to the group while it is locked.

Date Date 1 - 4 of 4

[integration-dev] [release] [genius-dev] Genius CSIT failing on master

Jamo Luhrsen <jluhrsen@...> #4702 patch merged toggle quoted message Show quoted text On 4/20/18 12:17 PM, Luis Gomez wrote: No way to verify in sandbox but this should fix: https://git.opendaylight.org/gerrit/#/c/71180 On Apr 20, 2018, at 11:17 AM, Luis Gomez <ecelgp@... <mailto:ecelgp@...>> wrote: OK, I am looking at this now, will update shortly. On Apr 20, 2018, at 10:20 AM, Sam Hague <shague@... <mailto:shague@...>> wrote: On Fri, Apr 20, 2018 at 9:46 AM, Vishal Thapar<vthapar@... <mailto:vthapar@...>>wrote: Without my patch compilaiton is failing [whole reason for the patch], so CSIT won't run. Ihad run the CSIT earlier, but ran into infra issues with space. Started it again. Vishal, how are you running csit? in your patch the csit failed the first time because the bundle_url was never set, or really was set to "to", so nothing is uploaded to nexus. Later when the csit runs, it doesn't have a distro so it has nothing to use. Maybe that is what you thought was a space issue. Luis is taking a look at the bundle_url problem as he thinks it has to do with the maven change. Regards, Vishal. On Fri, Apr 20, 2018 at 9:31 PM, Luis Gomez<ecelgp@... <mailto:ecelgp@...>>wrote: +1, this will determine whether the issue is related with the MDSAL recent change. On Apr 20, 2018, at 8:55 AM, Faseela K <faseela.k@... <mailto:faseela.k@...>> wrote: Vishal,____ I think it will be better to trigger a sandbox job for ovsdb CSIT with and without your patch, and see how it goes.____ Thanks,____ Faseela____ __ __ From:Vishal Thapar [mailto:vthapar@...] Sent:Friday, April 20, 2018 7:56 PM To:Faseela K <faseela.k@... <mailto:faseela.k@...>> Cc:Sam Hague <shague@... <mailto:shague@...>>; Swati Udhavrao Niture <swati.udhavrao.niture@... <mailto:swati.udhavrao.niture@...>>; K.V Suneelu Verma <k.v.suneelu.verma@... <mailto:k.v.suneelu.verma@...>>;integration-dev@... <mailto:integration-dev@...>;genius-dev@... <mailto:genius-dev@...>;ovsdb-dev@... <mailto:ovsdb-dev@...>;release@... <mailto:release@...>;netvirt-dev@... <mailto:netvirt-dev@...>;tsc@... <mailto:tsc@...> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ __ __ Hi Faseela,____ __ __ This sounds more and more related to the IpAddress issue. Can you try the patch I have on OVSDB and see if it works?____ __ __ Regards,____ Vishal.____ __ __ On Fri, Apr 20, 2018 at 7:18 PM, Faseela K <faseela.k@... <mailto:faseela.k@...>> wrote:____ Vishal/Sam,____ The current issue we are seeing is not due to bridge connect/disconnect.____ In a fresh clean setup also, tunnels are not getting configured with the DEBUG logs below.____ Do you think this can be due to the mdsal change, or something else? We don’t hit this issue in stable/oxygen.____ It is coming even in a local setup, with basic configuration.____ Thanks,____ Faseela____ ____ From:Vishal Thapar [mailto:vthapar@... <mailto:vthapar@...>] Sent:Friday, April 20, 2018 6:56 PM To:Sam Hague <shague@... <mailto:shague@...>> Cc:Swati Udhavrao Niture <swati.udhavrao.niture@... <mailto:swati.udhavrao.niture@...>>; Faseela K <faseela.k@... <mailto:faseela.k@...>>; K.V Suneelu Verma <k.v.suneelu.verma@... <mailto:k.v.suneelu.verma@...>>;integration-dev@... <mailto:integration-dev@...>;genius-dev@... <mailto:genius-dev@...>;ovsdb-dev@... <mailto:ovsdb-dev@...>;release@... <mailto:release@...>;netvirt-dev@... <mailto:netvirt-dev@...>____ Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ Hi Sam,____ ____ Yes, we need a similar patch in OVSDB and there was a discussion on this between Suneelu and Anil. The fix in HWVTEP doesn't apply as-is to OVSDB and we don't have a perfect solution to problem. The issue is seen when switch connects to only one node in cluster. Genius CSIT hardocded to ODL1 IP and I think HWVTEP issue was with HAPRoxy between TOR and ODL. Original mail thread is at [1].____ ____ Regards,____ Vishal.____ ____ [1]https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html <https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html>____ ____ On Fri, Apr 20, 2018 at 5:30 PM, Sam Hague <shague@... <mailto:shague@...>> wrote:____ ____ ____ On Fri, Apr 20, 2018 at 3:23 AM, Vishal Thapar <vthapar@... <mailto:vthapar@...>> wrote:____ I see log full of entries like this:____ 2018-04-20T14:33:20,490 \| DEBUG \| opendaylight-cluster-data-notification-dispatcher-56 \| OvsdbDataTreeChangeListener \| 418 - org.opendaylight.ovsdb.southbound-impl - 1.7.0.SNAPSHOT \|Did not find client for Node{getNodeId=Uri{_value=ovsdb://uuid/9fbcf6cc-e6e3-463b-8993-2a382963fb2b/bridge/br-int}, getTerminationPoint=[TerminationPoint{getTpId=Uri{_value=tunb1071e60d72}, augmentations={interface ____ ____ The upstream OVSBD CSIT doesn't have any exceptions, and the logging isn't at debug level so we won't see the above logs. I am pushing a patch so we can enable debugs there and have it also for troubleshooting.____ This may or may not be related to IpAddress issue in yangtools as connectionInfo uses Ipaddress.equals(). This could also be the old issue where stale connection hangs around. To rule the latter out, can we run only the ITM suite once? Or try one test case manually? Stale connection issue was caused by quick delete/add/delete of OVS connection. If we run the test case that doesn't do delete/add, which is current;y done at end of every suite, we'll come to know if it is yangtools/mdsal API issue or not.____ ____ If it is stale conneciton issue, share the logs with Suneelu, who can confirm if his proposed fix for OVSDB can address this, or just get him to share his WiP patch.____ Vishal, as an aside, the connection code is different for ovsdb and hwvtep, so we want to use the improvements from hwvtep also in ovsdb?____ ____ Regards,____ Vishal.____ ____ On Fri, Apr 20, 2018 at 2:46 PM, Swati Udhavrao Niture <swati.udhavrao.niture@... <mailto:swati.udhavrao.niture@...>> wrote:____ Hi Faseela,____ ____ PFA logs.____ ____ Thanks, Swati.____ ____ From:Faseela K Sent:Friday, April 20, 2018 12:05 PM To:Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma____ Cc:integration-dev@... <mailto:integration-dev@...>;genius-dev@... <mailto:genius-dev@...>;ovsdb-dev@... <mailto:ovsdb-dev@...>;release@... <mailto:release@...>;netvirt-dev@... <mailto:netvirt-dev@...> Subject:RE: [genius-dev] [release] Genius CSIT failing on master____ ____ Swati,____ Please pull the latest stable/oxygen, tunnel down issue is already resolved.____ Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.____ If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound and push tunnel configuration and supply the logs?____ Thanks,____ Faseela____ ____ From:Swati Udhavrao Niture Sent:Friday, April 20, 2018 12:03 PM To:Faseela K <faseela.k@... <mailto:faseela.k@...>>; Sam Hague <shague@... <mailto:shague@...>>; Vishal Thapar <vthapar@... <mailto:vthapar@...>>; K.V Suneelu Verma <k.v.suneelu.verma@... <mailto:k.v.suneelu.verma@...>> Cc:integration-dev@... <mailto:integration-dev@...>;genius-dev@... <mailto:genius-dev@...>;ovsdb-dev@... <mailto:ovsdb-dev@...>;release@... <mailto:release@...>;netvirt-dev@... <mailto:netvirt-dev@...> Subject:RE: [genius-dev] [release] Genius CSIT failing on master____ ____ Hi all,____ ____ Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request to bring up tunnels.____ Has anybody else faced the same issue?____ ____ Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.____ Any pointers on this?____ ____ Thanks, Swati.____ ____ From:genius-dev-bounces@... <mailto:genius-dev-bounces@...>[mailto:genius-dev-bounces@... <mailto:genius-dev-bounces@...>]On Behalf OfFaseela K Sent:Thursday, April 19, 2018 1:35 PM To:Sam Hague; Vishal Thapar; K.V Suneelu Verma Cc:integration-dev@... <mailto:integration-dev@...>;genius-dev@... <mailto:genius-dev@...>;ovsdb-dev@... <mailto:ovsdb-dev@...>;release@... <mailto:release@...> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ Yes, it is master.____ I hope the maven issues are all resolved now, but we are still hitting the same error.____ (karaf SSH issues are gone now, the ovsdb plugin issue still exists)____ ____ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz <https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz>____ ____ Thanks,____ Faseela____ ____ From:Sam Hague [mailto:shague@...] Sent:Wednesday, April 18, 2018 6:47 PM To:Vishal Thapar <vthapar@... <mailto:vthapar@...>> Cc:Faseela K <faseela.k@... <mailto:faseela.k@...>>;genius-dev@... <mailto:genius-dev@...>;integration-dev@... <mailto:integration-dev@...>;ovsdb-dev@... <mailto:ovsdb-dev@...>;release@... <mailto:release@...> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.____ ____ On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@... <mailto:vthapar@...>> wrote:____ No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.____ ____ Regards,____ Vishal.____ ____ On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@... <mailto:faseela.k@...>> wrote:____ +ovsdb-dev____ ____ Vishal,____ Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.____ But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change in ovsdb plugin?____ Thanks,____ Faseela____ ____ From:genius-dev-bounces@... <mailto:genius-dev-bounces@...>[mailto:genius-dev-bounces@... <mailto:genius-dev-bounces@...>]On Behalf OfB Sathwik Sent:Wednesday, April 18, 2018 3:48 PM To:release@... <mailto:release@...>;integration-dev@... <mailto:integration-dev@...> Cc:genius-dev@... <mailto:genius-dev@...> Subject:[genius-dev] Genius CSIT failing on master____ ____ Hi____ ____ Genius CSIT is Failing in master branch with some basic issues, Sometimes not able to login into karaf and sometime tunnels are not shown up on bridge side.____ ____ Since there are several distribution-check failures also seen intermittently on patches, wanted to know whether we have a stable master distribution, or____ We should wait to run CSIT later on.____ ____ Here are the logs of Genius-CSIT:____ ____ https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz>____ ____ https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz>____ ____ ____ ____ Best Regards____ Sathwik____ ____ _______________________________________________ release mailing list release@... <mailto:release@...> https://lists.opendaylight.org/mailman/listinfo/release <https://lists.opendaylight.org/mailman/listinfo/release>____ ____ _______________________________________________ genius-dev mailing list genius-dev@... <mailto:genius-dev@...> https://lists.opendaylight.org/mailman/listinfo/genius-dev <https://lists.opendaylight.org/mailman/listinfo/genius-dev>____ ____ ____ ____ __ __ _______________________________________________ release mailing list release@... <mailto:release@...> https://lists.opendaylight.org/mailman/listinfo/release <https://lists.opendaylight.org/mailman/listinfo/release> _______________________________________________ ovsdb-dev mailing list ovsdb-dev@... <mailto:ovsdb-dev@...> https://lists.opendaylight.org/mailman/listinfo/ovsdb-dev <https://lists.opendaylight.org/mailman/listinfo/ovsdb-dev> _______________________________________________ integration-dev mailing list integration-dev@... https://lists.opendaylight.org/mailman/listinfo/integration-dev
Vishal Thapar <vthapar@...> #4703 OVSDB CSIT seems to be passing with the patch. Thanks Sam for starting the run, didn't notice till after I started that it had already passed.. You're right I didn't interpret the logs correctly. So looks like the OVSDB fix works, but not sure what the plan is with MDSAL change, looks like it will be reverted. Sam/Jamo, Can we run Genius CSIT with this OVSDB patch? Regards, Vishal. toggle quoted message Show quoted text On Sat, Apr 21, 2018 at 2:34 AM, Jamo Luhrsen <jluhrsen@...> wrote: patch merged On 4/20/18 12:17 PM, Luis Gomez wrote: No way to verify in sandbox but this should fix: https://git.opendaylight.org/gerrit/#/c/71180 On Apr 20, 2018, at 11:17 AM, Luis Gomez <ecelgp@... <mailto:ecelgp@...>> wrote: OK, I am looking at this now, will update shortly. On Apr 20, 2018, at 10:20 AM, Sam Hague <shague@... <mailto:shague@...>> wrote: On Fri, Apr 20, 2018 at 9:46 AM, Vishal Thapar<vthapar@... <mailto:vthapar@...>>wrote: Without my patch compilaiton is failing [whole reason for the patch], so CSIT won't run. Ihad run the CSIT earlier, but ran into infra issues with space. Started it again. Vishal, how are you running csit? in your patch the csit failed the first time because the bundle_url was never set, or really was set to "to", so nothing is uploaded to nexus. Later when the csit runs, it doesn't have a distro so it has nothing to use. Maybe that is what you thought was a space issue. Luis is taking a look at the bundle_url problem as he thinks it has to do with the maven change. Regards, Vishal. On Fri, Apr 20, 2018 at 9:31 PM, Luis Gomez<ecelgp@... <mailto:ecelgp@...>>wrote: +1, this will determine whether the issue is related with the MDSAL recent change. On Apr 20, 2018, at 8:55 AM, Faseela K <faseela.k@... <mailto:faseela.k@...>> wrote: Vishal,____ I think it will be better to trigger a sandbox job for ovsdb CSIT with and without your patch, and see how it goes.____ Thanks,____ Faseela____ __ __ From:Vishal Thapar [mailto:vthapar@...] Sent:Friday, April 20, 2018 7:56 PM To:Faseela K <faseela.k@... <mailto:faseela.k@...>> Cc:Sam Hague <shague@... <mailto:shague@...>>; Swati Udhavrao Niture <swati.udhavrao.niture@ericsson.com <mailto:swati.udhavrao.niture@ericsson.com>>; K.V Suneelu Verma <k.v.suneelu.verma@...m <mailto:k.v.suneelu.verma@ericsson.com>>;integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org>;genius-dev@lists.opendaylight.org <mailto:genius-dev@...aylight.org>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org>;release@lists.opendaylight.org <mailto:release@...ight.org>;netvirt-dev@lists.opendaylight.org <mailto:netvirt-dev@...daylight.org>;tsc@...ylight.org <mailto:tsc@....org> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ __ __ Hi Faseela,____ __ __ This sounds more and more related to the IpAddress issue. Can you try the patch I have on OVSDB and see if it works?____ __ __ Regards,____ Vishal.____ __ __ On Fri, Apr 20, 2018 at 7:18 PM, Faseela K <faseela.k@... <mailto:faseela.k@...>> wrote:____ Vishal/Sam,____ The current issue we are seeing is not due to bridge connect/disconnect.____ In a fresh clean setup also, tunnels are not getting configured with the DEBUG logs below.____ Do you think this can be due to the mdsal change, or something else? We don’t hit this issue in stable/oxygen.____ It is coming even in a local setup, with basic configuration.____ Thanks,____ Faseela____ ____ From:Vishal Thapar [mailto:vthapar@... <mailto:vthapar@...>] Sent:Friday, April 20, 2018 6:56 PM To:Sam Hague <shague@... <mailto:shague@...>> Cc:Swati Udhavrao Niture <swati.udhavrao.niture@ericsson.com <mailto:swati.udhavrao.niture@ericsson.com>>; Faseela K <faseela.k@... <mailto:faseela.k@...>>; K.V Suneelu Verma <k.v.suneelu.verma@...m <mailto:k.v.suneelu.verma@ericsson.com>>;integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org>;genius-dev@lists.opendaylight.org <mailto:genius-dev@...aylight.org>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org>;release@lists.opendaylight.org <mailto:release@...ight.org>;netvirt-dev@lists.opendaylight.org <mailto:netvirt-dev@...daylight.org>____ Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ Hi Sam,____ ____ Yes, we need a similar patch in OVSDB and there was a discussion on this between Suneelu and Anil. The fix in HWVTEP doesn't apply as-is to OVSDB and we don't have a perfect solution to problem. The issue is seen when switch connects to only one node in cluster. Genius CSIT hardocded to ODL1 IP and I think HWVTEP issue was with HAPRoxy between TOR and ODL. Original mail thread is at [1].____ ____ Regards,____ Vishal.____ ____ [1]https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html <https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html>____ ____ On Fri, Apr 20, 2018 at 5:30 PM, Sam Hague <shague@... <mailto:shague@...>> wrote:____ ____ ____ On Fri, Apr 20, 2018 at 3:23 AM, Vishal Thapar <vthapar@... <mailto:vthapar@...>> wrote:____ I see log full of entries like this:____ 2018-04-20T14:33:20,490 \| DEBUG \| opendaylight-cluster-data-notification-dispatcher-56 \| OvsdbDataTreeChangeListener \| 418 - org.opendaylight.ovsdb.southbound-impl - 1.7.0.SNAPSHOT \|Did not find client for Node{getNodeId=Uri{_value=ovsdb://uuid/9fbcf6cc-e6e3-463b-8993-2a382963fb2b/bridge/br-int}, getTerminationPoint=[TerminationPoint{getTpId=Uri{_value=tunb1071e60d72}, augmentations={interface ____ ____ The upstream OVSBD CSIT doesn't have any exceptions, and the logging isn't at debug level so we won't see the above logs. I am pushing a patch so we can enable debugs there and have it also for troubleshooting.____ This may or may not be related to IpAddress issue in yangtools as connectionInfo uses Ipaddress.equals(). This could also be the old issue where stale connection hangs around. To rule the latter out, can we run only the ITM suite once? Or try one test case manually? Stale connection issue was caused by quick delete/add/delete of OVS connection. If we run the test case that doesn't do delete/add, which is current;y done at end of every suite, we'll come to know if it is yangtools/mdsal API issue or not.____ ____ If it is stale conneciton issue, share the logs with Suneelu, who can confirm if his proposed fix for OVSDB can address this, or just get him to share his WiP patch.____ Vishal, as an aside, the connection code is different for ovsdb and hwvtep, so we want to use the improvements from hwvtep also in ovsdb?____ ____ Regards,____ Vishal.____ ____ On Fri, Apr 20, 2018 at 2:46 PM, Swati Udhavrao Niture <swati.udhavrao.niture@ericsson.com <mailto:swati.udhavrao.niture@ericsson.com>> wrote:____ Hi Faseela,____ ____ PFA logs.____ ____ Thanks, Swati.____ ____ From:Faseela K Sent:Friday, April 20, 2018 12:05 PM To:Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma____ Cc:integration-dev@...ndaylight.org <mailto:integration-dev@lists.opendaylight.org>;genius-dev@lists.opendaylight.org <mailto:genius-dev@...aylight.org>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org>;release@lists.opendaylight.org <mailto:release@...ight.org>;netvirt-dev@lists.opendaylight.org <mailto:netvirt-dev@...daylight.org> Subject:RE: [genius-dev] [release] Genius CSIT failing on master____ ____ Swati,____ Please pull the latest stable/oxygen, tunnel down issue is already resolved.____ Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.____ If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound and push tunnel configuration and supply the logs?____ Thanks,____ Faseela____ ____ From:Swati Udhavrao Niture Sent:Friday, April 20, 2018 12:03 PM To:Faseela K <faseela.k@... <mailto:faseela.k@...>>; Sam Hague <shague@... <mailto:shague@...>>; Vishal Thapar <vthapar@... <mailto:vthapar@...>>; K.V Suneelu Verma <k.v.suneelu.verma@...m <mailto:k.v.suneelu.verma@ericsson.com>> Cc:integration-dev@...ndaylight.org <mailto:integration-dev@lists.opendaylight.org>;genius-dev@lists.opendaylight.org <mailto:genius-dev@...aylight.org>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org>;release@lists.opendaylight.org <mailto:release@...ight.org>;netvirt-dev@lists.opendaylight.org <mailto:netvirt-dev@...daylight.org> Subject:RE: [genius-dev] [release] Genius CSIT failing on master____ ____ Hi all,____ ____ Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request to bring up tunnels.____ Has anybody else faced the same issue?____ ____ Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.____ Any pointers on this?____ ____ Thanks, Swati.____ ____ From:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org>[mailto:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org>]On Behalf OfFaseela K Sent:Thursday, April 19, 2018 1:35 PM To:Sam Hague; Vishal Thapar; K.V Suneelu Verma Cc:integration-dev@...ndaylight.org <mailto:integration-dev@lists.opendaylight.org>;genius-dev@lists.opendaylight.org <mailto:genius-dev@...aylight.org>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org>;release@lists.opendaylight.org <mailto:release@...ight.org> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ Yes, it is master.____ I hope the maven issues are all resolved now, but we are still hitting the same error.____ (karaf SSH issues are gone now, the ovsdb plugin issue still exists)____ ____ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz <https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz>____ ____ Thanks,____ Faseela____ ____ From:Sam Hague [mailto:shague@...] Sent:Wednesday, April 18, 2018 6:47 PM To:Vishal Thapar <vthapar@... <mailto:vthapar@...>> Cc:Faseela K <faseela.k@... <mailto:faseela.k@...>>;genius-dev@...ht.org <mailto:genius-dev@...aylight.org>;integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org>;release@lists.opendaylight.org <mailto:release@...ight.org> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.____ ____ On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@... <mailto:vthapar@...>> wrote:____ No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.____ ____ Regards,____ Vishal.____ ____ On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@... <mailto:faseela.k@...>> wrote:____ +ovsdb-dev____ ____ Vishal,____ Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.____ But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change in ovsdb plugin?____ Thanks,____ Faseela____ ____ From:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org>[mailto:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org>]On Behalf OfB Sathwik Sent:Wednesday, April 18, 2018 3:48 PM To:release@...t.org <mailto:release@...ight.org>;integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org> Cc:genius-dev@...ight.org <mailto:genius-dev@...aylight.org> Subject:[genius-dev] Genius CSIT failing on master____ ____ Hi____ ____ Genius CSIT is Failing in master branch with some basic issues, Sometimes not able to login into karaf and sometime tunnels are not shown up on bridge side.____ ____ Since there are several distribution-check failures also seen intermittently on patches, wanted to know whether we have a stable master distribution, or____ We should wait to run CSIT later on.____ ____ Here are the logs of Genius-CSIT:____ ____ https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz>____ ____ https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz>____ ____ ____ ____ Best Regards____ Sathwik____ ____ _______________________________________________ release mailing list release@... <mailto:release@...ight.org> https://lists.opendaylight.org/mailman/listinfo/release <https://lists.opendaylight.org/mailman/listinfo/release>____ ____ _______________________________________________ genius-dev mailing list genius-dev@....org <mailto:genius-dev@...aylight.org> https://lists.opendaylight.org/mailman/listinfo/genius-dev <https://lists.opendaylight.org/mailman/listinfo/genius-dev>____ ____ ____ ____ __ __ _______________________________________________ release mailing list release@... <mailto:release@...ight.org> https://lists.opendaylight.org/mailman/listinfo/release <https://lists.opendaylight.org/mailman/listinfo/release> _______________________________________________ ovsdb-dev mailing list ovsdb-dev@...rg <mailto:ovsdb-dev@...ylight.org> https://lists.opendaylight.org/mailman/listinfo/ovsdb-dev <https://lists.opendaylight.org/mailman/listinfo/ovsdb-dev> _______________________________________________ integration-dev mailing list integration-dev@...ight.org https://lists.opendaylight.org/mailman/listinfo/integration-dev
Jamo Luhrsen <jluhrsen@...> #4704 Vishal, this patch, right? https://git.opendaylight.org/gerrit/#/c/71105/ if so, I used this from it's distro-check job: https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/ovsdb/karaf/0.9.0-SNAPSHOT/karaf-0.9.0-20180421.021919-68.zip to start this genius csit: https://jenkins.opendaylight.org/releng/view/genius/job/genius-csit-1node-gate-all-fluorine/34/ Thanks, JamO toggle quoted message Show quoted text On 4/20/18 7:54 PM, Vishal Thapar wrote: OVSDB CSIT seems to be passing with the patch. Thanks Sam for starting the run, didn't notice till after I started that it had already passed.. You're right I didn't interpret the logs correctly. So looks like the OVSDB fix works, but not sure what the plan is with MDSAL change, looks like it will be reverted. Sam/Jamo, Can we run Genius CSIT with this OVSDB patch? Regards, Vishal. On Sat, Apr 21, 2018 at 2:34 AM, Jamo Luhrsen <jluhrsen@... <mailto:jluhrsen@...>> wrote: patch merged On 4/20/18 12:17 PM, Luis Gomez wrote: No way to verify in sandbox but this should fix: https://git.opendaylight.org/gerrit/#/c/71180 <https://git.opendaylight.org/gerrit/#/c/71180> On Apr 20, 2018, at 11:17 AM, Luis Gomez <ecelgp@... <mailto:ecelgp@...> <mailto:ecelgp@... <mailto:ecelgp@...>>> wrote: OK, I am looking at this now, will update shortly. On Apr 20, 2018, at 10:20 AM, Sam Hague <shague@... <mailto:shague@...> <mailto:shague@... <mailto:shague@...>>> wrote: On Fri, Apr 20, 2018 at 9:46 AM, Vishal Thapar<vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>>wrote: Without my patch compilaiton is failing [whole reason for the patch], so CSIT won't run. Ihad run the CSIT earlier, but ran into infra issues with space. Started it again. Vishal, how are you running csit? in your patch the csit failed the first time because the bundle_url was never set, or really was set to "to", so nothing is uploaded to nexus. Later when the csit runs, it doesn't have a distro so it has nothing to use. Maybe that is what you thought was a space issue. Luis is taking a look at the bundle_url problem as he thinks it has to do with the maven change. Regards, Vishal. On Fri, Apr 20, 2018 at 9:31 PM, Luis Gomez<ecelgp@... <mailto:ecelgp@...> <mailto:ecelgp@... <mailto:ecelgp@...>>>wrote: +1, this will determine whether the issue is related with the MDSAL recent change. On Apr 20, 2018, at 8:55 AM, Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>> wrote: Vishal,____ I think it will be better to trigger a sandbox job for ovsdb CSIT with and without your patch, and see how it goes.____ Thanks,____ Faseela____ __ __ From:Vishal Thapar [mailto:vthapar@... <mailto:vthapar@...>] Sent:Friday, April 20, 2018 7:56 PM To:Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>> Cc:Sam Hague <shague@... <mailto:shague@...> <mailto:shague@... <mailto:shague@...>>>; Swati Udhavrao Niture <swati.udhavrao.niture@... <mailto:swati.udhavrao.niture@...> <mailto:swati.udhavrao.niture@... <mailto:swati.udhavrao.niture@...>>>; K.V Suneelu Verma <k.v.suneelu.verma@... <mailto:k.v.suneelu.verma@...> <mailto:k.v.suneelu.verma@... <mailto:k.v.suneelu.verma@...>>>;integration-dev@... <mailto:integration-dev@...> <mailto:integration-dev@... <mailto:integration-dev@...>>;genius-dev@... <mailto:genius-dev@...> <mailto:genius-dev@... <mailto:genius-dev@...>>;ovsdb-dev@... <mailto:ovsdb-dev@...> <mailto:ovsdb-dev@... <mailto:ovsdb-dev@...>>;release@... <mailto:release@...> <mailto:release@... <mailto:release@...>>;netvirt-dev@... <mailto:netvirt-dev@...> <mailto:netvirt-dev@... <mailto:netvirt-dev@...>>;tsc@... <mailto:tsc@...> <mailto:tsc@... <mailto:tsc@...>> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ __ __ Hi Faseela,____ __ __ This sounds more and more related to the IpAddress issue. Can you try the patch I have on OVSDB and see if it works?____ __ __ Regards,____ Vishal.____ __ __ On Fri, Apr 20, 2018 at 7:18 PM, Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>> wrote:____ Vishal/Sam,____ The current issue we are seeing is not due to bridge connect/disconnect.____ In a fresh clean setup also, tunnels are not getting configured with the DEBUG logs below.____ Do you think this can be due to the mdsal change, or something else? We don’t hit this issue in stable/oxygen.____ It is coming even in a local setup, with basic configuration.____ Thanks,____ Faseela____ ____ From:Vishal Thapar [mailto:vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>] Sent:Friday, April 20, 2018 6:56 PM To:Sam Hague <shague@... <mailto:shague@...> <mailto:shague@... <mailto:shague@...>>> Cc:Swati Udhavrao Niture <swati.udhavrao.niture@... <mailto:swati.udhavrao.niture@...> <mailto:swati.udhavrao.niture@... <mailto:swati.udhavrao.niture@...>>>; Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>>; K.V Suneelu Verma <k.v.suneelu.verma@... <mailto:k.v.suneelu.verma@...> <mailto:k.v.suneelu.verma@... <mailto:k.v.suneelu.verma@...>>>;integration-dev@... <mailto:integration-dev@...> <mailto:integration-dev@... <mailto:integration-dev@...>>;genius-dev@... <mailto:genius-dev@...> <mailto:genius-dev@... <mailto:genius-dev@...>>;ovsdb-dev@... <mailto:ovsdb-dev@...> <mailto:ovsdb-dev@... <mailto:ovsdb-dev@...>>;release@... <mailto:release@...> <mailto:release@... <mailto:release@...>>;netvirt-dev@... <mailto:netvirt-dev@...> <mailto:netvirt-dev@... <mailto:netvirt-dev@...>>____ Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ Hi Sam,____ ____ Yes, we need a similar patch in OVSDB and there was a discussion on this between Suneelu and Anil. The fix in HWVTEP doesn't apply as-is to OVSDB and we don't have a perfect solution to problem. The issue is seen when switch connects to only one node in cluster. Genius CSIT hardocded to ODL1 IP and I think HWVTEP issue was with HAPRoxy between TOR and ODL. Original mail thread is at [1].____ ____ Regards,____ Vishal.____ ____ [1]https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html <https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html> <https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html <https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html>>____ ____ On Fri, Apr 20, 2018 at 5:30 PM, Sam Hague <shague@... <mailto:shague@...> <mailto:shague@... <mailto:shague@...>>> wrote:____ ____ ____ On Fri, Apr 20, 2018 at 3:23 AM, Vishal Thapar <vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>> wrote:____ I see log full of entries like this:____ 2018-04-20T14:33:20,490 \| DEBUG \| opendaylight-cluster-data-notification-dispatcher-56 \| OvsdbDataTreeChangeListener \| 418 - org.opendaylight.ovsdb.southbound-impl - 1.7.0.SNAPSHOT \|Did not find client for Node{getNodeId=Uri{_value=ovsdb://uuid/9fbcf6cc-e6e3-463b-8993-2a382963fb2b/bridge/br-int}, getTerminationPoint=[TerminationPoint{getTpId=Uri{_value=tunb1071e60d72}, augmentations={interface ____ ____ The upstream OVSBD CSIT doesn't have any exceptions, and the logging isn't at debug level so we won't see the above logs. I am pushing a patch so we can enable debugs there and have it also for troubleshooting.____ This may or may not be related to IpAddress issue in yangtools as connectionInfo uses Ipaddress.equals(). This could also be the old issue where stale connection hangs around. To rule the latter out, can we run only the ITM suite once? Or try one test case manually? Stale connection issue was caused by quick delete/add/delete of OVS connection. If we run the test case that doesn't do delete/add, which is current;y done at end of every suite, we'll come to know if it is yangtools/mdsal API issue or not.____ ____ If it is stale conneciton issue, share the logs with Suneelu, who can confirm if his proposed fix for OVSDB can address this, or just get him to share his WiP patch.____ Vishal, as an aside, the connection code is different for ovsdb and hwvtep, so we want to use the improvements from hwvtep also in ovsdb?____ ____ Regards,____ Vishal.____ ____ On Fri, Apr 20, 2018 at 2:46 PM, Swati Udhavrao Niture <swati.udhavrao.niture@... <mailto:swati.udhavrao.niture@...> <mailto:swati.udhavrao.niture@... <mailto:swati.udhavrao.niture@...>>> wrote:____ Hi Faseela,____ ____ PFA logs.____ ____ Thanks, Swati.____ ____ From:Faseela K Sent:Friday, April 20, 2018 12:05 PM To:Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma____ Cc:integration-dev@... <mailto:integration-dev@...> <mailto:integration-dev@... <mailto:integration-dev@...>>;genius-dev@... <mailto:genius-dev@...> <mailto:genius-dev@... <mailto:genius-dev@...>>;ovsdb-dev@... <mailto:ovsdb-dev@...> <mailto:ovsdb-dev@... <mailto:ovsdb-dev@...>>;release@... <mailto:release@...> <mailto:release@... <mailto:release@...>>;netvirt-dev@... <mailto:netvirt-dev@...> <mailto:netvirt-dev@... <mailto:netvirt-dev@...>> Subject:RE: [genius-dev] [release] Genius CSIT failing on master____ ____ Swati,____ Please pull the latest stable/oxygen, tunnel down issue is already resolved.____ Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.____ If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound and push tunnel configuration and supply the logs?____ Thanks,____ Faseela____ ____ From:Swati Udhavrao Niture Sent:Friday, April 20, 2018 12:03 PM To:Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>>; Sam Hague <shague@... <mailto:shague@...> <mailto:shague@... <mailto:shague@...>>>; Vishal Thapar <vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>>; K.V Suneelu Verma <k.v.suneelu.verma@... <mailto:k.v.suneelu.verma@...> <mailto:k.v.suneelu.verma@... <mailto:k.v.suneelu.verma@...>>> Cc:integration-dev@... <mailto:integration-dev@...> <mailto:integration-dev@... <mailto:integration-dev@...>>;genius-dev@... <mailto:genius-dev@...> <mailto:genius-dev@... <mailto:genius-dev@...>>;ovsdb-dev@... <mailto:ovsdb-dev@...> <mailto:ovsdb-dev@... <mailto:ovsdb-dev@...>>;release@... <mailto:release@...> <mailto:release@... <mailto:release@...>>;netvirt-dev@... <mailto:netvirt-dev@...> <mailto:netvirt-dev@... <mailto:netvirt-dev@...>> Subject:RE: [genius-dev] [release] Genius CSIT failing on master____ ____ Hi all,____ ____ Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request to bring up tunnels.____ Has anybody else faced the same issue?____ ____ Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.____ Any pointers on this?____ ____ Thanks, Swati.____ ____ From:genius-dev-bounces@... <mailto:genius-dev-bounces@...> <mailto:genius-dev-bounces@... <mailto:genius-dev-bounces@...>>[mailto:genius-dev-bounces@... <mailto:genius-dev-bounces@...> <mailto:genius-dev-bounces@... <mailto:genius-dev-bounces@...>>]On Behalf OfFaseela K Sent:Thursday, April 19, 2018 1:35 PM To:Sam Hague; Vishal Thapar; K.V Suneelu Verma Cc:integration-dev@... <mailto:integration-dev@...> <mailto:integration-dev@... <mailto:integration-dev@...>>;genius-dev@... <mailto:genius-dev@...> <mailto:genius-dev@... <mailto:genius-dev@...>>;ovsdb-dev@... <mailto:ovsdb-dev@...> <mailto:ovsdb-dev@... <mailto:ovsdb-dev@...>>;release@... <mailto:release@...> <mailto:release@... <mailto:release@...>> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ Yes, it is master.____ I hope the maven issues are all resolved now, but we are still hitting the same error.____ (karaf SSH issues are gone now, the ovsdb plugin issue still exists)____ ____ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz <https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz> <https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz <https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz>>____ ____ Thanks,____ Faseela____ ____ From:Sam Hague [mailto:shague@... <mailto:shague@...>] Sent:Wednesday, April 18, 2018 6:47 PM To:Vishal Thapar <vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>> Cc:Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>>;genius-dev@... <mailto:genius-dev@...> <mailto:genius-dev@... <mailto:genius-dev@...>>;integration-dev@... <mailto:integration-dev@...> <mailto:integration-dev@... <mailto:integration-dev@...>>;ovsdb-dev@... <mailto:ovsdb-dev@...> <mailto:ovsdb-dev@... <mailto:ovsdb-dev@...>>;release@... <mailto:release@...> <mailto:release@... <mailto:release@...>> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.____ ____ On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>> wrote:____ No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.____ ____ Regards,____ Vishal.____ ____ On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>> wrote:____ +ovsdb-dev____ ____ Vishal,____ Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.____ But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change in ovsdb plugin?____ Thanks,____ Faseela____ ____ From:genius-dev-bounces@... <mailto:genius-dev-bounces@...> <mailto:genius-dev-bounces@... <mailto:genius-dev-bounces@...>>[mailto:genius-dev-bounces@... <mailto:genius-dev-bounces@...> <mailto:genius-dev-bounces@... <mailto:genius-dev-bounces@...>>]On Behalf OfB Sathwik Sent:Wednesday, April 18, 2018 3:48 PM To:release@... <mailto:release@...> <mailto:release@... <mailto:release@...>>;integration-dev@... <mailto:integration-dev@...> <mailto:integration-dev@... <mailto:integration-dev@...>> Cc:genius-dev@... <mailto:genius-dev@...> <mailto:genius-dev@... <mailto:genius-dev@...>> Subject:[genius-dev] Genius CSIT failing on master____ ____ Hi____ ____ Genius CSIT is Failing in master branch with some basic issues, Sometimes not able to login into karaf and sometime tunnels are not shown up on bridge side.____ ____ Since there are several distribution-check failures also seen intermittently on patches, wanted to know whether we have a stable master distribution, or____ We should wait to run CSIT later on.____ ____ Here are the logs of Genius-CSIT:____ ____ https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz> <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz>>____ ____ https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz> <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz>>____ ____ ____ ____ Best Regards____ Sathwik____ ____ _______________________________________________ release mailing list release@... <mailto:release@...> <mailto:release@... <mailto:release@...>> https://lists.opendaylight.org/mailman/listinfo/release <https://lists.opendaylight.org/mailman/listinfo/release> <https://lists.opendaylight.org/mailman/listinfo/release <https://lists.opendaylight.org/mailman/listinfo/release>>____ ____ _______________________________________________ genius-dev mailing list genius-dev@... <mailto:genius-dev@...> <mailto:genius-dev@... <mailto:genius-dev@...>> https://lists.opendaylight.org/mailman/listinfo/genius-dev <https://lists.opendaylight.org/mailman/listinfo/genius-dev> <https://lists.opendaylight.org/mailman/listinfo/genius-dev <https://lists.opendaylight.org/mailman/listinfo/genius-dev>>____ ____ ____ ____ __ __ _______________________________________________ release mailing list release@... <mailto:release@...> <mailto:release@... <mailto:release@...>> https://lists.opendaylight.org/mailman/listinfo/release <https://lists.opendaylight.org/mailman/listinfo/release> <https://lists.opendaylight.org/mailman/listinfo/release <https://lists.opendaylight.org/mailman/listinfo/release>> _______________________________________________ ovsdb-dev mailing list ovsdb-dev@... <mailto:ovsdb-dev@...> <mailto:ovsdb-dev@... <mailto:ovsdb-dev@...>> https://lists.opendaylight.org/mailman/listinfo/ovsdb-dev <https://lists.opendaylight.org/mailman/listinfo/ovsdb-dev> <https://lists.opendaylight.org/mailman/listinfo/ovsdb-dev <https://lists.opendaylight.org/mailman/listinfo/ovsdb-dev>> _______________________________________________ integration-dev mailing list integration-dev@... <mailto:integration-dev@...> https://lists.opendaylight.org/mailman/listinfo/integration-dev <https://lists.opendaylight.org/mailman/listinfo/integration-dev>
Vishal Thapar <vthapar@...> #4705 Thanks Jamo, and looks like Genius CSIT did pass. Regards, Vishal. P.S. Trimmed the To list. toggle quoted message Show quoted text On Sat, Apr 21, 2018 at 11:15 AM, Jamo Luhrsen <jluhrsen@...> wrote: Vishal, this patch, right? https://git.opendaylight.org/gerrit/#/c/71105/ if so, I used this from it's distro-check job: https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/ovsdb/karaf/0.9.0-SNAPSHOT/karaf-0.9.0-20180421.021919-68.zip to start this genius csit: https://jenkins.opendaylight.org/releng/view/genius/job/genius-csit-1node-gate-all-fluorine/34/ Thanks, JamO On 4/20/18 7:54 PM, Vishal Thapar wrote: OVSDB CSIT seems to be passing with the patch. Thanks Sam for starting the run, didn't notice till after I started that it had already passed.. You're right I didn't interpret the logs correctly. So looks like the OVSDB fix works, but not sure what the plan is with MDSAL change, looks like it will be reverted. Sam/Jamo, Can we run Genius CSIT with this OVSDB patch? Regards, Vishal. On Sat, Apr 21, 2018 at 2:34 AM, Jamo Luhrsen <jluhrsen@... <mailto:jluhrsen@...>> wrote: patch merged On 4/20/18 12:17 PM, Luis Gomez wrote: No way to verify in sandbox but this should fix: https://git.opendaylight.org/gerrit/#/c/71180 <https://git.opendaylight.org/gerrit/#/c/71180> On Apr 20, 2018, at 11:17 AM, Luis Gomez <ecelgp@... <mailto:ecelgp@...> <mailto:ecelgp@... <mailto:ecelgp@...>>> wrote: OK, I am looking at this now, will update shortly. On Apr 20, 2018, at 10:20 AM, Sam Hague <shague@... <mailto:shague@...> <mailto:shague@... <mailto:shague@...>>> wrote: On Fri, Apr 20, 2018 at 9:46 AM, Vishal Thapar<vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>>wrote: Without my patch compilaiton is failing [whole reason for the patch], so CSIT won't run. Ihad run the CSIT earlier, but ran into infra issues with space. Started it again. Vishal, how are you running csit? in your patch the csit failed the first time because the bundle_url was never set, or really was set to "to", so nothing is uploaded to nexus. Later when the csit runs, it doesn't have a distro so it has nothing to use. Maybe that is what you thought was a space issue. Luis is taking a look at the bundle_url problem as he thinks it has to do with the maven change. Regards, Vishal. On Fri, Apr 20, 2018 at 9:31 PM, Luis Gomez<ecelgp@... <mailto:ecelgp@...> <mailto:ecelgp@... <mailto:ecelgp@...>>>wrote: +1, this will determine whether the issue is related with the MDSAL recent change. On Apr 20, 2018, at 8:55 AM, Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>> wrote: Vishal,____ I think it will be better to trigger a sandbox job for ovsdb CSIT with and without your patch, and see how it goes.____ Thanks,____ Faseela____ __ __ From:Vishal Thapar [mailto:vthapar@... <mailto:vthapar@...>] Sent:Friday, April 20, 2018 7:56 PM To:Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>> Cc:Sam Hague <shague@... <mailto:shague@...> <mailto:shague@... <mailto:shague@...>>>; Swati Udhavrao Niture <swati.udhavrao.niture@ericsson.com <mailto:swati.udhavrao.niture@ericsson.com> <mailto:swati.udhavrao.niture@ericsson.com <mailto:swati.udhavrao.niture@ericsson.com>>>; K.V Suneelu Verma <k.v.suneelu.verma@...m <mailto:k.v.suneelu.verma@ericsson.com> <mailto:k.v.suneelu.verma@ericsson.com <mailto:k.v.suneelu.verma@ericsson.com>>>;integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org> <mailto:integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org>>;genius-dev@lists.opendaylight.org <mailto:genius-dev@...aylight.org> <mailto:genius-dev@...aylight.org <mailto:genius-dev@...aylight.org>>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org> <mailto:ovsdb-dev@...ylight.org <mailto:ovsdb-dev@...ylight.org>>;release@lists.opendaylight.org <mailto:release@...ight.org> <mailto:release@...ight.org <mailto:release@...ight.org>>;netvirt-dev@lists.opendaylight.org <mailto:netvirt-dev@...daylight.org> <mailto:netvirt-dev@...daylight.org <mailto:netvirt-dev@...daylight.org>>;tsc@...aylight.org <mailto:tsc@....org> <mailto:tsc@....org <mailto:tsc@....org>> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ __ __ Hi Faseela,____ __ __ This sounds more and more related to the IpAddress issue. Can you try the patch I have on OVSDB and see if it works?____ __ __ Regards,____ Vishal.____ __ __ On Fri, Apr 20, 2018 at 7:18 PM, Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>> wrote:____ Vishal/Sam,____ The current issue we are seeing is not due to bridge connect/disconnect.____ In a fresh clean setup also, tunnels are not getting configured with the DEBUG logs below.____ Do you think this can be due to the mdsal change, or something else? We don’t hit this issue in stable/oxygen.____ It is coming even in a local setup, with basic configuration.____ Thanks,____ Faseela____ ____ From:Vishal Thapar [mailto:vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>] Sent:Friday, April 20, 2018 6:56 PM To:Sam Hague <shague@... <mailto:shague@...> <mailto:shague@... <mailto:shague@...>>> Cc:Swati Udhavrao Niture <swati.udhavrao.niture@ericsson.com <mailto:swati.udhavrao.niture@ericsson.com> <mailto:swati.udhavrao.niture@ericsson.com <mailto:swati.udhavrao.niture@ericsson.com>>>; Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>>; K.V Suneelu Verma <k.v.suneelu.verma@...m <mailto:k.v.suneelu.verma@ericsson.com> <mailto:k.v.suneelu.verma@ericsson.com <mailto:k.v.suneelu.verma@ericsson.com>>>;integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org> <mailto:integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org>>;genius-dev@lists.opendaylight.org <mailto:genius-dev@...aylight.org> <mailto:genius-dev@...aylight.org <mailto:genius-dev@...aylight.org>>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org> <mailto:ovsdb-dev@...ylight.org <mailto:ovsdb-dev@...ylight.org>>;release@lists.opendaylight.org <mailto:release@...ight.org> <mailto:release@...ight.org <mailto:release@...ight.org>>;netvirt-dev@lists.opendaylight.org <mailto:netvirt-dev@...daylight.org> <mailto:netvirt-dev@...daylight.org <mailto:netvirt-dev@...daylight.org>>____ Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ Hi Sam,____ ____ Yes, we need a similar patch in OVSDB and there was a discussion on this between Suneelu and Anil. The fix in HWVTEP doesn't apply as-is to OVSDB and we don't have a perfect solution to problem. The issue is seen when switch connects to only one node in cluster. Genius CSIT hardocded to ODL1 IP and I think HWVTEP issue was with HAPRoxy between TOR and ODL. Original mail thread is at [1].____ ____ Regards,____ Vishal.____ ____ [1]https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html <https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html> <https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html <https://lists.opendaylight.org/pipermail/ovsdb-dev/2018-February/004567.html>>____ ____ On Fri, Apr 20, 2018 at 5:30 PM, Sam Hague <shague@... <mailto:shague@...> <mailto:shague@... <mailto:shague@...>>> wrote:____ ____ ____ On Fri, Apr 20, 2018 at 3:23 AM, Vishal Thapar <vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>> wrote:____ I see log full of entries like this:____ 2018-04-20T14:33:20,490 \| DEBUG \| opendaylight-cluster-data-notification-dispatcher-56 \| OvsdbDataTreeChangeListener \| 418 - org.opendaylight.ovsdb.southbound-impl - 1.7.0.SNAPSHOT \|Did not find client for Node{getNodeId=Uri{_value=ovsdb://uuid/9fbcf6cc-e6e3-463b-8993-2a382963fb2b/bridge/br-int}, getTerminationPoint=[TerminationPoint{getTpId=Uri{_value=tunb1071e60d72}, augmentations={interface ____ ____ The upstream OVSBD CSIT doesn't have any exceptions, and the logging isn't at debug level so we won't see the above logs. I am pushing a patch so we can enable debugs there and have it also for troubleshooting.____ This may or may not be related to IpAddress issue in yangtools as connectionInfo uses Ipaddress.equals(). This could also be the old issue where stale connection hangs around. To rule the latter out, can we run only the ITM suite once? Or try one test case manually? Stale connection issue was caused by quick delete/add/delete of OVS connection. If we run the test case that doesn't do delete/add, which is current;y done at end of every suite, we'll come to know if it is yangtools/mdsal API issue or not.____ ____ If it is stale conneciton issue, share the logs with Suneelu, who can confirm if his proposed fix for OVSDB can address this, or just get him to share his WiP patch.____ Vishal, as an aside, the connection code is different for ovsdb and hwvtep, so we want to use the improvements from hwvtep also in ovsdb?____ ____ Regards,____ Vishal.____ ____ On Fri, Apr 20, 2018 at 2:46 PM, Swati Udhavrao Niture <swati.udhavrao.niture@ericsson.com <mailto:swati.udhavrao.niture@ericsson.com> <mailto:swati.udhavrao.niture@ericsson.com <mailto:swati.udhavrao.niture@ericsson.com>>> wrote:____ Hi Faseela,____ ____ PFA logs.____ ____ Thanks, Swati.____ ____ From:Faseela K Sent:Friday, April 20, 2018 12:05 PM To:Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma____ Cc:integration-dev@...ndaylight.org <mailto:integration-dev@lists.opendaylight.org> <mailto:integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org>>;genius-dev@lists.opendaylight.org <mailto:genius-dev@...aylight.org> <mailto:genius-dev@...aylight.org <mailto:genius-dev@...aylight.org>>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org> <mailto:ovsdb-dev@...ylight.org <mailto:ovsdb-dev@...ylight.org>>;release@lists.opendaylight.org <mailto:release@...ight.org> <mailto:release@...ight.org <mailto:release@...ight.org>>;netvirt-dev@lists.opendaylight.org <mailto:netvirt-dev@...daylight.org> <mailto:netvirt-dev@...daylight.org <mailto:netvirt-dev@...daylight.org>> Subject:RE: [genius-dev] [release] Genius CSIT failing on master____ ____ Swati,____ Please pull the latest stable/oxygen, tunnel down issue is already resolved.____ Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.____ If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound and push tunnel configuration and supply the logs?____ Thanks,____ Faseela____ ____ From:Swati Udhavrao Niture Sent:Friday, April 20, 2018 12:03 PM To:Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>>; Sam Hague <shague@... <mailto:shague@...> <mailto:shague@... <mailto:shague@...>>>; Vishal Thapar <vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>>; K.V Suneelu Verma <k.v.suneelu.verma@...m <mailto:k.v.suneelu.verma@ericsson.com> <mailto:k.v.suneelu.verma@ericsson.com <mailto:k.v.suneelu.verma@ericsson.com>>> Cc:integration-dev@...ndaylight.org <mailto:integration-dev@lists.opendaylight.org> <mailto:integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org>>;genius-dev@lists.opendaylight.org <mailto:genius-dev@...aylight.org> <mailto:genius-dev@...aylight.org <mailto:genius-dev@...aylight.org>>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org> <mailto:ovsdb-dev@...ylight.org <mailto:ovsdb-dev@...ylight.org>>;release@lists.opendaylight.org <mailto:release@...ight.org> <mailto:release@...ight.org <mailto:release@...ight.org>>;netvirt-dev@lists.opendaylight.org <mailto:netvirt-dev@...daylight.org> <mailto:netvirt-dev@...daylight.org <mailto:netvirt-dev@...daylight.org>> Subject:RE: [genius-dev] [release] Genius CSIT failing on master____ ____ Hi all,____ ____ Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request to bring up tunnels.____ Has anybody else faced the same issue?____ ____ Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.____ Any pointers on this?____ ____ Thanks, Swati.____ ____ From:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org> <mailto:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org>>[mailto:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org> <mailto:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org>>]On Behalf OfFaseela K Sent:Thursday, April 19, 2018 1:35 PM To:Sam Hague; Vishal Thapar; K.V Suneelu Verma Cc:integration-dev@...ndaylight.org <mailto:integration-dev@lists.opendaylight.org> <mailto:integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org>>;genius-dev@lists.opendaylight.org <mailto:genius-dev@...aylight.org> <mailto:genius-dev@...aylight.org <mailto:genius-dev@...aylight.org>>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org> <mailto:ovsdb-dev@...ylight.org <mailto:ovsdb-dev@...ylight.org>>;release@lists.opendaylight.org <mailto:release@...ight.org> <mailto:release@...ight.org <mailto:release@...ight.org>> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ Yes, it is master.____ I hope the maven issues are all resolved now, but we are still hitting the same error.____ (karaf SSH issues are gone now, the ovsdb plugin issue still exists)____ ____ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz <https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz> <https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz <https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz>>____ ____ Thanks,____ Faseela____ ____ From:Sam Hague [mailto:shague@... <mailto:shague@...>] Sent:Wednesday, April 18, 2018 6:47 PM To:Vishal Thapar <vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>> Cc:Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>>;genius-dev@...ght.org <mailto:genius-dev@...aylight.org> <mailto:genius-dev@...aylight.org <mailto:genius-dev@...aylight.org>>;integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org> <mailto:integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org>>;ovsdb-dev@lists.opendaylight.org <mailto:ovsdb-dev@...ylight.org> <mailto:ovsdb-dev@...ylight.org <mailto:ovsdb-dev@...ylight.org>>;release@lists.opendaylight.org <mailto:release@...ight.org> <mailto:release@...ight.org <mailto:release@...ight.org>> Subject:Re: [genius-dev] [release] Genius CSIT failing on master____ ____ If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.____ ____ On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@... <mailto:vthapar@...> <mailto:vthapar@... <mailto:vthapar@...>>> wrote:____ No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.____ ____ Regards,____ Vishal.____ ____ On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@... <mailto:faseela.k@...> <mailto:faseela.k@... <mailto:faseela.k@...>>> wrote:____ +ovsdb-dev____ ____ Vishal,____ Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.____ But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change in ovsdb plugin?____ Thanks,____ Faseela____ ____ From:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org> <mailto:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org>>[mailto:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org> <mailto:genius-dev-bounces@lists.opendaylight.org <mailto:genius-dev-bounces@lists.opendaylight.org>>]On Behalf OfB Sathwik Sent:Wednesday, April 18, 2018 3:48 PM To:release@...t.org <mailto:release@...ight.org> <mailto:release@...ight.org <mailto:release@...ight.org>>;integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org> <mailto:integration-dev@lists.opendaylight.org <mailto:integration-dev@lists.opendaylight.org>> Cc:genius-dev@...ight.org <mailto:genius-dev@...aylight.org> <mailto:genius-dev@...aylight.org <mailto:genius-dev@...aylight.org>> Subject:[genius-dev] Genius CSIT failing on master____ ____ Hi____ ____ Genius CSIT is Failing in master branch with some basic issues, Sometimes not able to login into karaf and sometime tunnels are not shown up on bridge side.____ ____ Since there are several distribution-check failures also seen intermittently on patches, wanted to know whether we have a stable master distribution, or____ We should wait to run CSIT later on.____ ____ Here are the logs of Genius-CSIT:____ ____ https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz> <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz>>____ ____ https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz> <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz <https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz>>____ ____ ____ ____ Best Regards____ Sathwik____ ____ _______________________________________________ release mailing list release@... <mailto:release@...ight.org> <mailto:release@...ight.org <mailto:release@...ight.org>> https://lists.opendaylight.org/mailman/listinfo/release <https://lists.opendaylight.org/mailman/listinfo/release> <https://lists.opendaylight.org/mailman/listinfo/release <https://lists.opendaylight.org/mailman/listinfo/release>>____ ____ _______________________________________________ genius-dev mailing list genius-dev@....org <mailto:genius-dev@...aylight.org> <mailto:genius-dev@...aylight.org <mailto:genius-dev@...aylight.org>> https://lists.opendaylight.org/mailman/listinfo/genius-dev <https://lists.opendaylight.org/mailman/listinfo/genius-dev> <https://lists.opendaylight.org/mailman/listinfo/genius-dev <https://lists.opendaylight.org/mailman/listinfo/genius-dev>>____ ____ ____ ____ __ __ _______________________________________________ release mailing list release@... <mailto:release@...ight.org> <mailto:release@...ight.org <mailto:release@...ight.org>> https://lists.opendaylight.org/mailman/listinfo/release <https://lists.opendaylight.org/mailman/listinfo/release> <https://lists.opendaylight.org/mailman/listinfo/release

1 - 4 of 4

1

Previous Topic Next Topic

Home Hashtags Subgroups Terms