Re: [genius-dev] [release] Genius CSIT failing on master


Faseela K
 

Vishal,

   I think it will be better to trigger a sandbox job for ovsdb CSIT with and without your patch, and see how it goes.

Thanks,

Faseela

 

From: Vishal Thapar [mailto:vthapar@...]
Sent: Friday, April 20, 2018 7:56 PM
To: Faseela K <faseela.k@...>
Cc: Sam Hague <shague@...>; Swati Udhavrao Niture <swati.udhavrao.niture@...>; K.V Suneelu Verma <k.v.suneelu.verma@...>; integration-dev@...; genius-dev@...; ovsdb-dev@...; release@...; netvirt-dev@...; tsc@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master

 

Hi Faseela,

 

This sounds more and more related to the IpAddress issue. Can you try the patch I have on OVSDB and see if it works?

 

Regards,

Vishal.

 

On Fri, Apr 20, 2018 at 7:18 PM, Faseela K <faseela.k@...> wrote:

Vishal/Sam,

   The current issue we are seeing is not due to bridge connect/disconnect.

   In a fresh clean setup also, tunnels are not getting configured with the DEBUG logs below.

   Do you think this can be due to the mdsal change, or something else? We don’t hit this issue in stable/oxygen.

   It is coming even in a local setup, with basic configuration.

Thanks,

Faseela

 

From: Vishal Thapar [mailto:vthapar@...]
Sent: Friday, April 20, 2018 6:56 PM
To: Sam Hague <shague@...>
Cc: Swati Udhavrao Niture <swati.udhavrao.niture@...>; Faseela K <faseela.k@...>; K.V Suneelu Verma <k.v.suneelu.verma@...>; integration-dev@...; genius-dev@...; ovsdb-dev@...; release@...; netvirt-dev@...


Subject: Re: [genius-dev] [release] Genius CSIT failing on master

 

Hi Sam,

 

Yes, we need a similar patch in OVSDB and there was a discussion on this between Suneelu and Anil. The fix in HWVTEP doesn't apply as-is to OVSDB and we don't have a perfect solution to problem. The issue is seen when switch connects to only one node in cluster. Genius CSIT hardocded to ODL1 IP and I think HWVTEP issue was with HAPRoxy between TOR and ODL. Original mail thread is at [1].

 

Regards,

Vishal.

 

 

On Fri, Apr 20, 2018 at 5:30 PM, Sam Hague <shague@...> wrote:

 

 

On Fri, Apr 20, 2018 at 3:23 AM, Vishal Thapar <vthapar@...> wrote:

I see log full of entries like this:

2018-04-20T14:33:20,490 | DEBUG | opendaylight-cluster-data-notification-dispatcher-56 | OvsdbDataTreeChangeListener      | 418 - org.opendaylight.ovsdb.southbound-impl - 1.7.0.SNAPSHOT | Did not find client for Node{getNodeId=Uri{_value=ovsdb://uuid/9fbcf6cc-e6e3-463b-8993-2a382963fb2b/bridge/br-int}, getTerminationPoint=[TerminationPoint{getTpId=Uri{_value=tunb1071e60d72}, augmentations={interface 

 

The upstream OVSBD CSIT doesn't have any exceptions, and the logging isn't at debug level so we won't see the above logs. I am pushing a patch so we can enable debugs there and have it also for troubleshooting.

This may or may not be related to IpAddress issue in yangtools as connectionInfo uses Ipaddress.equals(). This could also be the old issue where stale connection hangs around. To rule the latter out, can we run only the ITM suite once? Or try one test case manually? Stale connection issue was caused by quick delete/add/delete of OVS connection. If we run the test case that doesn't do delete/add, which is current;y done at end of every suite, we'll come to know if it is yangtools/mdsal API issue or not.

 

If it is stale conneciton issue, share the logs with Suneelu, who can confirm if his proposed fix for OVSDB can address this, or just get him to share his WiP patch.

Vishal, as an aside, the connection code is different for ovsdb and hwvtep, so we want to use the improvements from hwvtep also in ovsdb?

 

Regards,

Vishal.

 

On Fri, Apr 20, 2018 at 2:46 PM, Swati Udhavrao Niture <swati.udhavrao.niture@...> wrote:

Hi Faseela,

 

PFA logs.

 

Thanks,
Swati.

 

From: Faseela K
Sent: Friday, April 20, 2018 12:05 PM
To: Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma


Cc: integration-dev@...; genius-dev@...; ovsdb-dev@...; release@...; netvirt-dev@...
Subject: RE: [genius-dev] [release] Genius CSIT failing on master

 

Swati,

   Please pull the latest stable/oxygen, tunnel down issue is already resolved.

   Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.

   If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound and push tunnel configuration and supply the logs?

Thanks,

Faseela

 

From: Swati Udhavrao Niture
Sent: Friday, April 20, 2018 12:03 PM
To: Faseela K <faseela.k@...>; Sam Hague <shague@...>; Vishal Thapar <vthapar@...>; K.V Suneelu Verma <k.v.suneelu.verma@...>
Cc: integration-dev@...; genius-dev@...; ovsdb-dev@...; release@...; netvirt-dev@...
Subject: RE: [genius-dev] [release] Genius CSIT failing on master

 

Hi all,

 

Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request to bring up tunnels.

Has anybody else faced the same issue?

 

Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.

Any pointers on this?

 

Thanks,
Swati.

 

From: genius-dev-bounces@... [mailto:genius-dev-bounces@...] On Behalf Of Faseela K
Sent: Thursday, April 19, 2018 1:35 PM
To: Sam Hague; Vishal Thapar; K.V Suneelu Verma
Cc: integration-dev@...; genius-dev@...; ovsdb-dev@...; release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master

 

Yes, it is master.

I hope the maven issues are all resolved now, but we are still hitting the same error.

(karaf SSH issues are gone now, the ovsdb plugin issue still exists)

 

https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz

 

Thanks,

Faseela

 

From: Sam Hague [mailto:shague@...]
Sent: Wednesday, April 18, 2018 6:47 PM
To: Vishal Thapar <vthapar@...>
Cc: Faseela K <faseela.k@...>; genius-dev@...; integration-dev@...; ovsdb-dev@...; release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master

 

If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.

 

On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@...> wrote:

No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.

 

Regards,

Vishal.

 

On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@...> wrote:

+ovsdb-dev

 

Vishal,

  Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.

  But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change in ovsdb plugin?

Thanks,

Faseela

 

From: genius-dev-bounces@... [mailto:genius-dev-bounces@...] On Behalf Of B Sathwik
Sent: Wednesday, April 18, 2018 3:48 PM
To: release@...; integration-dev@...
Cc: genius-dev@...
Subject: [genius-dev] Genius CSIT failing on master

 

Hi

 

Genius CSIT is Failing in master branch with some basic issues, Sometimes not able to login into karaf and sometime tunnels are not shown up on bridge side.

 

Since there are several distribution-check failures also seen intermittently on patches, wanted to know whether we have a stable master distribution, or

We should wait to run CSIT later on.

 

Here are the logs of Genius-CSIT:

 

https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-1node-gate-all-fluorine/4/robot-plugin/log.html.gz

 

https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/sathwikgenius-csit-3node-gate-all-fluorine/3/robot-plugin/log.html.gz

 

 

 

Best Regards

Sathwik

 


_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release

 

_______________________________________________
genius-dev mailing list
genius-dev@...
https://lists.opendaylight.org/mailman/listinfo/genius-dev

 

 

 

 

Join z.archive.ovsdb-dev@lists.opendaylight.org to automatically receive all group messages.