|
If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.
On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar < vthapar@...> wrote: No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.
Regards, Vishal.
genius-dev mailing list
genius-dev@...
https://lists.opendaylight.org/mailman/listinfo/genius-dev
|
|
|
toggle quoted message
Show quoted text
From: Sam Hague [mailto:shague@...]
Sent: Wednesday, April 18, 2018 6:47 PM
To: Vishal Thapar <vthapar@...>
Cc: Faseela K <faseela.k@...>; genius-dev@...; integration-dev@...; ovsdb-dev@...; release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.
On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@...> wrote:
No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had a
similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.
On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@...> wrote:
+ovsdb-dev
Vishal,
Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.
But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change
in ovsdb plugin?
Thanks,
Faseela
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release
_______________________________________________
genius-dev mailing list
genius-dev@...
https://lists.opendaylight.org/mailman/listinfo/genius-dev
|
|
|
Swati Udhavrao Niture <swati.udhavrao.niture@...>
Hi all,
Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request to bring up tunnels.
Has anybody else faced the same issue?
Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.
Any pointers on this?
Thanks,
Swati.
toggle quoted message
Show quoted text
From: genius-dev-bounces@... [mailto:genius-dev-bounces@...]
On Behalf Of Faseela K
Sent: Thursday, April 19, 2018 1:35 PM
To: Sam Hague; Vishal Thapar; K.V Suneelu Verma
Cc: integration-dev@...; genius-dev@...; ovsdb-dev@...; release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
Yes, it is master.
I hope the maven issues are all resolved now, but we are still hitting the same error.
(karaf SSH issues are gone now, the ovsdb plugin issue still exists)
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz
Thanks,
Faseela
From: Sam Hague [mailto:shague@...]
Sent: Wednesday, April 18, 2018 6:47 PM
To: Vishal Thapar <vthapar@...>
Cc: Faseela K <faseela.k@...>;
genius-dev@...;
integration-dev@...;
ovsdb-dev@...;
release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.
On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@...> wrote:
No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had
a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.
On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@...> wrote:
+ovsdb-dev
Vishal,
Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.
But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change
in ovsdb plugin?
Thanks,
Faseela
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release
_______________________________________________
genius-dev mailing list
genius-dev@...
https://lists.opendaylight.org/mailman/listinfo/genius-dev
|
|
|
Swati,
Please pull the latest stable/oxygen, tunnel down issue is already resolved.
Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.
If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound and push tunnel configuration and supply the logs?
Thanks,
Faseela
toggle quoted message
Show quoted text
From: Swati Udhavrao Niture
Sent: Friday, April 20, 2018 12:03 PM
To: Faseela K <faseela.k@...>; Sam Hague <shague@...>; Vishal Thapar <vthapar@...>; K.V Suneelu Verma <k.v.suneelu.verma@...>
Cc: integration-dev@...; genius-dev@...; ovsdb-dev@...; release@...; netvirt-dev@...
Subject: RE: [genius-dev] [release] Genius CSIT failing on master
Hi all,
Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request to bring up tunnels.
Has anybody else faced the same issue?
Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.
Any pointers on this?
Thanks,
Swati.
Yes, it is master.
I hope the maven issues are all resolved now, but we are still hitting the same error.
(karaf SSH issues are gone now, the ovsdb plugin issue still exists)
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz
Thanks,
Faseela
From: Sam Hague [mailto:shague@...]
Sent: Wednesday, April 18, 2018 6:47 PM
To: Vishal Thapar <vthapar@...>
Cc: Faseela K <faseela.k@...>;
genius-dev@...;
integration-dev@...;
ovsdb-dev@...;
release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.
On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@...> wrote:
No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had a
similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.
On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@...> wrote:
+ovsdb-dev
Vishal,
Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.
But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change
in ovsdb plugin?
Thanks,
Faseela
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release
_______________________________________________
genius-dev mailing list
genius-dev@...
https://lists.opendaylight.org/mailman/listinfo/genius-dev
|
|
|
Swati Udhavrao Niture <swati.udhavrao.niture@...>
Hi Faseela,
Ok, sure.
Thanks,
Swati.
toggle quoted message
Show quoted text
From: Faseela K
Sent: Friday, April 20, 2018 12:05 PM
To: Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma
Cc: integration-dev@...; genius-dev@...; ovsdb-dev@...; release@...; netvirt-dev@...
Subject: RE: [genius-dev] [release] Genius CSIT failing on master
Swati,
Please pull the latest stable/oxygen, tunnel down issue is already resolved.
Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.
If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound and push tunnel configuration and supply the logs?
Thanks,
Faseela
Hi all,
Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request to bring up tunnels.
Has anybody else faced the same issue?
Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.
Any pointers on this?
Thanks,
Swati.
Yes, it is master.
I hope the maven issues are all resolved now, but we are still hitting the same error.
(karaf SSH issues are gone now, the ovsdb plugin issue still exists)
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz
Thanks,
Faseela
From: Sam Hague [mailto:shague@...]
Sent: Wednesday, April 18, 2018 6:47 PM
To: Vishal Thapar <vthapar@...>
Cc: Faseela K <faseela.k@...>;
genius-dev@...;
integration-dev@...;
ovsdb-dev@...;
release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.
On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@...> wrote:
No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had
a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.
On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@...> wrote:
+ovsdb-dev
Vishal,
Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.
But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change
in ovsdb plugin?
Thanks,
Faseela
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release
_______________________________________________
genius-dev mailing list
genius-dev@...
https://lists.opendaylight.org/mailman/listinfo/genius-dev
|
|
|
Swati Udhavrao Niture <swati.udhavrao.niture@...>
Hi Faseela,
PFA logs.
Thanks,
Swati.
toggle quoted message
Show quoted text
From: Faseela K
Sent: Friday, April 20, 2018 12:05 PM
To: Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma
Cc: integration-dev@...; genius-dev@...; ovsdb-dev@...; release@...; netvirt-dev@...
Subject: RE: [genius-dev] [release] Genius CSIT failing on master
Swati,
Please pull the latest stable/oxygen, tunnel down issue is already resolved.
Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.
If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound and push tunnel configuration and supply the logs?
Thanks,
Faseela
Hi all,
Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request to bring up tunnels.
Has anybody else faced the same issue?
Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.
Any pointers on this?
Thanks,
Swati.
Yes, it is master.
I hope the maven issues are all resolved now, but we are still hitting the same error.
(karaf SSH issues are gone now, the ovsdb plugin issue still exists)
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz
Thanks,
Faseela
From: Sam Hague [mailto:shague@...]
Sent: Wednesday, April 18, 2018 6:47 PM
To: Vishal Thapar <vthapar@...>
Cc: Faseela K <faseela.k@...>;
genius-dev@...;
integration-dev@...;
ovsdb-dev@...;
release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.
On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@...> wrote:
No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had
a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.
On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@...> wrote:
+ovsdb-dev
Vishal,
Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.
But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change
in ovsdb plugin?
Thanks,
Faseela
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release
_______________________________________________
genius-dev mailing list
genius-dev@...
https://lists.opendaylight.org/mailman/listinfo/genius-dev
|
|
|
Vishal Thapar <vthapar@...>
I see log full of entries like this: 2018-04-20T14:33:20,490 | DEBUG | opendaylight-cluster-data-notification-dispatcher-56 | OvsdbDataTreeChangeListener | 418 - org.opendaylight.ovsdb.southbound-impl - 1.7.0.SNAPSHOT | Did not find client for Node{getNodeId=Uri{_value=ovsdb://uuid/9fbcf6cc-e6e3-463b-8993-2a382963fb2b/bridge/br-int}, getTerminationPoint=[TerminationPoint{getTpId=Uri{_value=tunb1071e60d72}, augmentations={interface
This may or may not be related to IpAddress issue in yangtools as connectionInfo uses Ipaddress.equals(). This could also be the old issue where stale connection hangs around. To rule the latter out, can we run only the ITM suite once? Or try one test case manually? Stale connection issue was caused by quick delete/add/delete of OVS connection. If we run the test case that doesn't do delete/add, which is current;y done at end of every suite, we'll come to know if it is yangtools/mdsal API issue or not.
If it is stale conneciton issue, share the logs with Suneelu, who can confirm if his proposed fix for OVSDB can address this, or just get him to share his WiP patch.
Regards, Vishal.
On Fri, Apr 20, 2018 at 2:46 PM, Swati Udhavrao Niture <swati.udhavrao.niture@...> wrote:
Hi Faseela,
PFA logs.
Thanks,
Swati.
From: Faseela K
Sent: Friday, April 20, 2018 12:05 PM
To: Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma
Swati,
Please pull the latest stable/oxygen, tunnel down issue is already resolved.
Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.
If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound and push tunnel configuration and supply the logs?
Thanks,
Faseela
Hi all,
Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request to bring up tunnels.
Has anybody else faced the same issue?
Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.
Any pointers on this?
Thanks,
Swati.
Yes, it is master.
I hope the maven issues are all resolved now, but we are still hitting the same error.
(karaf SSH issues are gone now, the ovsdb plugin issue still exists)
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz
Thanks,
Faseela
From: Sam Hague [mailto:shague@...]
Sent: Wednesday, April 18, 2018 6:47 PM
To: Vishal Thapar <vthapar@...>
Cc: Faseela K <faseela.k@...>;
genius-dev@....org;
integration-dev@lists.opendaylight.org;
ovsdb-dev@....org;
release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.
On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@...> wrote:
No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup on OVS restart, could be related to that. Suneelu had
a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for Suneelu too.
On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@...> wrote:
+ovsdb-dev
Vishal,
Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.
But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change
in ovsdb plugin?
Thanks,
Faseela
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release
_______________________________________________
genius-dev mailing list
genius-dev@....org
https://lists.opendaylight.org/mailman/listinfo/genius-dev
|
|
|
Swati Udhavrao Niture <swati.udhavrao.niture@...>
Hi Vishal,
I brought up controller after cleaning data, journal and snapshots.
After this, I pushed tunnel configurations. But tunnel is not coming on the switch.
The TRACE logs were captured during this operation. There were no switch connect/disconnects or create/delete involved.
Thanks,
Swati.
toggle quoted message
Show quoted text
From: Vishal Thapar [mailto:vthapar@...]
Sent: Friday, April 20, 2018 3:53 PM
To: Swati Udhavrao Niture
Cc: Faseela K; Sam Hague; K.V Suneelu Verma; integration-dev@...; genius-dev@...; ovsdb-dev@...; release@...; netvirt-dev@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
I see log full of entries like this:
2018-04-20T14:33:20,490 | DEBUG | opendaylight-cluster-data-notification-dispatcher-56 | OvsdbDataTreeChangeListener | 418 - org.opendaylight.ovsdb.southbound-impl - 1.7.0.SNAPSHOT |
Did not find client for Node{getNodeId=Uri{_value=ovsdb://uuid/9fbcf6cc-e6e3-463b-8993-2a382963fb2b/bridge/br-int}, getTerminationPoint=[TerminationPoint{getTpId=Uri{_value=tunb1071e60d72}, augmentations={interface
This may or may not be related to IpAddress issue in yangtools as connectionInfo uses Ipaddress.equals(). This could also be the old issue where stale connection hangs around. To rule the
latter out, can we run only the ITM suite once? Or try one test case manually? Stale connection issue was caused by quick delete/add/delete of OVS connection. If we run the test case that doesn't do delete/add, which is current;y done at end of every suite,
we'll come to know if it is yangtools/mdsal API issue or not.
If it is stale conneciton issue, share the logs with Suneelu, who can confirm if his proposed fix for OVSDB can address this, or just get him to share his WiP patch.
On Fri, Apr 20, 2018 at 2:46 PM, Swati Udhavrao Niture <swati.udhavrao.niture@...> wrote:
Hi Faseela,
PFA logs.
Thanks,
Swati.
From: Faseela K
Sent: Friday, April 20, 2018 12:05 PM
To: Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma
Swati,
Please pull the latest stable/oxygen, tunnel down issue is already resolved.
Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.
If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound
and push tunnel configuration and supply the logs?
Thanks,
Faseela
Hi all,
Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request
to bring up tunnels.
Has anybody else faced the same issue?
Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.
Any pointers on this?
Thanks,
Swati.
Yes, it is master.
I hope the maven issues are all resolved now, but we are still hitting the same error.
(karaf SSH issues are gone now, the ovsdb plugin issue still exists)
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz
Thanks,
Faseela
From: Sam Hague [mailto:shague@...]
Sent: Wednesday, April 18, 2018 6:47 PM
To: Vishal Thapar <vthapar@...>
Cc: Faseela K <faseela.k@...>;
genius-dev@...;
integration-dev@...;
ovsdb-dev@...;
release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.
On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@...> wrote:
No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup
on OVS restart, could be related to that. Suneelu had a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for
Suneelu too.
On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@...> wrote:
+ovsdb-dev
Vishal,
Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.
But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change
in ovsdb plugin?
Thanks,
Faseela
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release
_______________________________________________
genius-dev mailing list
genius-dev@...
https://lists.opendaylight.org/mailman/listinfo/genius-dev
|
|
|
Vishal, as an aside, the connection code is different for ovsdb and hwvtep, so we want to use the improvements from hwvtep also in ovsdb?
|
|
|
Vishal Thapar <vthapar@...>
Hi Sam,
Yes, we need a similar patch in OVSDB and there was a discussion on this between Suneelu and Anil. The fix in HWVTEP doesn't apply as-is to OVSDB and we don't have a perfect solution to problem. The issue is seen when switch connects to only one node in cluster. Genius CSIT hardocded to ODL1 IP and I think HWVTEP issue was with HAPRoxy between TOR and ODL. Original mail thread is at [1].
Regards, Vishal.
On Fri, Apr 20, 2018 at 5:30 PM, Sam Hague <shague@...> wrote:
Vishal, as an aside, the connection code is different for ovsdb and hwvtep, so we want to use the improvements from hwvtep also in ovsdb?
|
|
|
Vishal/Sam,
The current issue we are seeing is not due to bridge connect/disconnect.
In a fresh clean setup also, tunnels are not getting configured with the DEBUG logs below.
Do you think this can be due to the mdsal change, or something else? We don’t hit this issue in stable/oxygen.
It is coming even in a local setup, with basic configuration.
Thanks,
Faseela
toggle quoted message
Show quoted text
From: Vishal Thapar [mailto:vthapar@...]
Sent: Friday, April 20, 2018 6:56 PM
To: Sam Hague <shague@...>
Cc: Swati Udhavrao Niture <swati.udhavrao.niture@...>; Faseela K <faseela.k@...>; K.V Suneelu Verma <k.v.suneelu.verma@...>; integration-dev@...; genius-dev@...; ovsdb-dev@...;
release@...; netvirt-dev@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
Yes, we need a similar patch in OVSDB and there was a discussion on this between Suneelu and Anil. The fix in HWVTEP doesn't apply as-is to OVSDB and we don't have a perfect solution to problem.
The issue is seen when switch connects to only one node in cluster. Genius CSIT hardocded to ODL1 IP and I think HWVTEP issue was with HAPRoxy between TOR and ODL. Original mail thread is at [1].
On Fri, Apr 20, 2018 at 5:30 PM, Sam Hague <shague@...> wrote:
On Fri, Apr 20, 2018 at 3:23 AM, Vishal Thapar <vthapar@...> wrote:
I see log full of entries like this:
2018-04-20T14:33:20,490 | DEBUG | opendaylight-cluster-data-notification-dispatcher-56 | OvsdbDataTreeChangeListener | 418 - org.opendaylight.ovsdb.southbound-impl - 1.7.0.SNAPSHOT |
Did not find client for Node{getNodeId=Uri{_value=ovsdb://uuid/9fbcf6cc-e6e3-463b-8993-2a382963fb2b/bridge/br-int}, getTerminationPoint=[TerminationPoint{getTpId=Uri{_value=tunb1071e60d72}, augmentations={interface
The upstream OVSBD CSIT doesn't have any exceptions, and the logging isn't at debug level so we won't see the above logs. I am pushing a patch so we can enable debugs there and have it also for troubleshooting.
This may or may not be related to IpAddress issue in yangtools as connectionInfo uses Ipaddress.equals(). This could also be the old issue where stale connection hangs around. To rule the latter
out, can we run only the ITM suite once? Or try one test case manually? Stale connection issue was caused by quick delete/add/delete of OVS connection. If we run the test case that doesn't do delete/add, which is current;y done at end of every suite, we'll
come to know if it is yangtools/mdsal API issue or not.
If it is stale conneciton issue, share the logs with Suneelu, who can confirm if his proposed fix for OVSDB can address this, or just get him to share his WiP patch.
Vishal, as an aside, the connection code is different for ovsdb and hwvtep, so we want to use the improvements from hwvtep also in ovsdb?
On Fri, Apr 20, 2018 at 2:46 PM, Swati Udhavrao Niture <swati.udhavrao.niture@...> wrote:
Hi Faseela,
PFA logs.
Thanks,
Swati.
From: Faseela K
Sent: Friday, April 20, 2018 12:05 PM
To: Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma
Swati,
Please pull the latest stable/oxygen, tunnel down issue is already resolved.
Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.
If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound
and push tunnel configuration and supply the logs?
Thanks,
Faseela
Hi all,
Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request
to bring up tunnels.
Has anybody else faced the same issue?
Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.
Any pointers on this?
Thanks,
Swati.
Yes, it is master.
I hope the maven issues are all resolved now, but we are still hitting the same error.
(karaf SSH issues are gone now, the ovsdb plugin issue still exists)
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz
Thanks,
Faseela
From: Sam Hague [mailto:shague@...]
Sent: Wednesday, April 18, 2018 6:47 PM
To: Vishal Thapar <vthapar@...>
Cc: Faseela K <faseela.k@...>;
genius-dev@...;
integration-dev@...;
ovsdb-dev@...;
release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.
On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@...> wrote:
No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup
on OVS restart, could be related to that. Suneelu had a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for
Suneelu too.
On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@...> wrote:
+ovsdb-dev
Vishal,
Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.
But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change
in ovsdb plugin?
Thanks,
Faseela
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release
_______________________________________________
genius-dev mailing list
genius-dev@...
https://lists.opendaylight.org/mailman/listinfo/genius-dev
|
|
|
Vishal Thapar <vthapar@...>
Hi Faseela,
This sounds more and more related to the IpAddress issue. Can you try the patch I have on OVSDB and see if it works?
Regards, Vishal.
On Fri, Apr 20, 2018 at 7:18 PM, Faseela K <faseela.k@...> wrote:
Vishal/Sam,
The current issue we are seeing is not due to bridge connect/disconnect.
In a fresh clean setup also, tunnels are not getting configured with the DEBUG logs below.
Do you think this can be due to the mdsal change, or something else? We don’t hit this issue in stable/oxygen.
It is coming even in a local setup, with basic configuration.
Thanks,
Faseela
From: Vishal Thapar [mailto:vthapar@...]
Sent: Friday, April 20, 2018 6:56 PM
To: Sam Hague <shague@...>
Cc: Swati Udhavrao Niture <swati.udhavrao.niture@ericsson.com>; Faseela K <faseela.k@...>; K.V Suneelu Verma <k.v.suneelu.verma@ericsson.com>; integration-dev@lists.opendaylight.org; genius-dev@....org; ovsdb-dev@....org;
release@...; netvirt-dev@lists.opendaylight.org
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
Yes, we need a similar patch in OVSDB and there was a discussion on this between Suneelu and Anil. The fix in HWVTEP doesn't apply as-is to OVSDB and we don't have a perfect solution to problem.
The issue is seen when switch connects to only one node in cluster. Genius CSIT hardocded to ODL1 IP and I think HWVTEP issue was with HAPRoxy between TOR and ODL. Original mail thread is at [1].
On Fri, Apr 20, 2018 at 5:30 PM, Sam Hague <shague@...> wrote:
On Fri, Apr 20, 2018 at 3:23 AM, Vishal Thapar <vthapar@...> wrote:
I see log full of entries like this:
2018-04-20T14:33:20,490 | DEBUG | opendaylight-cluster-data-notification-dispatcher-56 | OvsdbDataTreeChangeListener | 418 - org.opendaylight.ovsdb.southbound-impl - 1.7.0.SNAPSHOT |
Did not find client for Node{getNodeId=Uri{_value=ovsdb://uuid/9fbcf6cc-e6e3-463b-8993-2a382963fb2b/bridge/br-int}, getTerminationPoint=[TerminationPoint{getTpId=Uri{_value=tunb1071e60d72}, augmentations={interface
The upstream OVSBD CSIT doesn't have any exceptions, and the logging isn't at debug level so we won't see the above logs. I am pushing a patch so we can enable debugs there and have it also for troubleshooting.
This may or may not be related to IpAddress issue in yangtools as connectionInfo uses Ipaddress.equals(). This could also be the old issue where stale connection hangs around. To rule the latter
out, can we run only the ITM suite once? Or try one test case manually? Stale connection issue was caused by quick delete/add/delete of OVS connection. If we run the test case that doesn't do delete/add, which is current;y done at end of every suite, we'll
come to know if it is yangtools/mdsal API issue or not.
If it is stale conneciton issue, share the logs with Suneelu, who can confirm if his proposed fix for OVSDB can address this, or just get him to share his WiP patch.
Vishal, as an aside, the connection code is different for ovsdb and hwvtep, so we want to use the improvements from hwvtep also in ovsdb?
On Fri, Apr 20, 2018 at 2:46 PM, Swati Udhavrao Niture <swati.udhavrao.niture@ericsson.com> wrote:
Hi Faseela,
PFA logs.
Thanks,
Swati.
From: Faseela K
Sent: Friday, April 20, 2018 12:05 PM
To: Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma
Swati,
Please pull the latest stable/oxygen, tunnel down issue is already resolved.
Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.
If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound
and push tunnel configuration and supply the logs?
Thanks,
Faseela
Hi all,
Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request
to bring up tunnels.
Has anybody else faced the same issue?
Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.
Any pointers on this?
Thanks,
Swati.
Yes, it is master.
I hope the maven issues are all resolved now, but we are still hitting the same error.
(karaf SSH issues are gone now, the ovsdb plugin issue still exists)
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz
Thanks,
Faseela
From: Sam Hague [mailto:shague@...]
Sent: Wednesday, April 18, 2018 6:47 PM
To: Vishal Thapar <vthapar@...>
Cc: Faseela K <faseela.k@...>;
genius-dev@....org;
integration-dev@lists.opendaylight.org;
ovsdb-dev@....org;
release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.
On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@...> wrote:
No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup
on OVS restart, could be related to that. Suneelu had a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for
Suneelu too.
On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@...> wrote:
+ovsdb-dev
Vishal,
Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.
But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change
in ovsdb plugin?
Thanks,
Faseela
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release
_______________________________________________
genius-dev mailing list
genius-dev@....org
https://lists.opendaylight.org/mailman/listinfo/genius-dev
|
|
|
Vishal,
I think it will be better to trigger a sandbox job for ovsdb CSIT with and without your patch, and see how it goes.
Thanks,
Faseela
toggle quoted message
Show quoted text
From: Vishal Thapar [mailto:vthapar@...]
Sent: Friday, April 20, 2018 7:56 PM
To: Faseela K <faseela.k@...>
Cc: Sam Hague <shague@...>; Swati Udhavrao Niture <swati.udhavrao.niture@...>; K.V Suneelu Verma <k.v.suneelu.verma@...>; integration-dev@...; genius-dev@...; ovsdb-dev@...;
release@...; netvirt-dev@...; tsc@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
This sounds more and more related to the IpAddress issue. Can you try the patch I have on OVSDB and see if it works?
On Fri, Apr 20, 2018 at 7:18 PM, Faseela K <faseela.k@...> wrote:
Vishal/Sam,
The current issue we are seeing is not due to bridge connect/disconnect.
In a fresh clean setup also, tunnels are not getting configured with the DEBUG logs below.
Do you think this can be due to the mdsal change, or something else? We don’t hit this issue in stable/oxygen.
It is coming even in a local setup, with basic configuration.
Thanks,
Faseela
From: Vishal Thapar [mailto:vthapar@...]
Sent: Friday, April 20, 2018 6:56 PM
To: Sam Hague <shague@...>
Cc: Swati Udhavrao Niture <swati.udhavrao.niture@...>; Faseela K <faseela.k@...>; K.V Suneelu Verma
<k.v.suneelu.verma@...>;
integration-dev@...;
genius-dev@...;
ovsdb-dev@...;
release@...;
netvirt-dev@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
Yes, we need a similar patch in OVSDB and there was a discussion on this between Suneelu and Anil. The fix in HWVTEP doesn't apply
as-is to OVSDB and we don't have a perfect solution to problem. The issue is seen when switch connects to only one node in cluster. Genius CSIT hardocded to ODL1 IP and I think HWVTEP issue was with HAPRoxy between TOR and ODL. Original mail thread is at [1].
On Fri, Apr 20, 2018 at 5:30 PM, Sam Hague <shague@...> wrote:
On Fri, Apr 20, 2018 at 3:23 AM, Vishal Thapar <vthapar@...> wrote:
I see log full of entries like this:
2018-04-20T14:33:20,490 | DEBUG | opendaylight-cluster-data-notification-dispatcher-56 | OvsdbDataTreeChangeListener | 418 -
org.opendaylight.ovsdb.southbound-impl - 1.7.0.SNAPSHOT | Did not find client for Node{getNodeId=Uri{_value=ovsdb://uuid/9fbcf6cc-e6e3-463b-8993-2a382963fb2b/bridge/br-int}, getTerminationPoint=[TerminationPoint{getTpId=Uri{_value=tunb1071e60d72}, augmentations={interface
The upstream OVSBD CSIT doesn't have any exceptions, and the logging isn't at debug level so we won't see the above logs. I am pushing a patch so we can enable debugs there and
have it also for troubleshooting.
This may or may not be related to IpAddress issue in yangtools as connectionInfo uses Ipaddress.equals(). This could also be the old
issue where stale connection hangs around. To rule the latter out, can we run only the ITM suite once? Or try one test case manually? Stale connection issue was caused by quick delete/add/delete of OVS connection. If we run the test case that doesn't do delete/add,
which is current;y done at end of every suite, we'll come to know if it is yangtools/mdsal API issue or not.
If it is stale conneciton issue, share the logs with Suneelu, who can confirm if his proposed fix for OVSDB can address this, or just
get him to share his WiP patch.
Vishal, as an aside, the connection code is different for ovsdb and hwvtep, so we want to use the improvements from hwvtep also in ovsdb?
On Fri, Apr 20, 2018 at 2:46 PM, Swati Udhavrao Niture <swati.udhavrao.niture@...> wrote:
Hi Faseela,
PFA logs.
Thanks,
Swati.
From: Faseela K
Sent: Friday, April 20, 2018 12:05 PM
To: Swati Udhavrao Niture; Sam Hague; Vishal Thapar; K.V Suneelu Verma
Swati,
Please pull the latest stable/oxygen, tunnel down issue is already resolved.
Master has some issues with ovsdb plugin, which we are still debugging, and no handle yet.
If you are working on latest master, could you please enable TRACE logs for org.opendaylight.ovsdb.southbound
and push tunnel configuration and supply the logs?
Thanks,
Faseela
Hi all,
Tunnels are coming up on the switch now, but I have to restart my OVS every-time after I send a request
to bring up tunnels.
Has anybody else faced the same issue?
Adding to this, in stable-oxygen, tunnels are coming up on the switch, but I see oper_status as down.
Any pointers on this?
Thanks,
Swati.
Yes, it is master.
I hope the maven issues are all resolved now, but we are still hitting the same error.
(karaf SSH issues are gone now, the ovsdb plugin issue still exists)
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-1node-gate-all-fluorine/33/robot-plugin/log.html.gz
Thanks,
Faseela
From: Sam Hague [mailto:shague@...]
Sent: Wednesday, April 18, 2018 6:47 PM
To: Vishal Thapar <vthapar@...>
Cc: Faseela K <faseela.k@...>;
genius-dev@...;
integration-dev@...;
ovsdb-dev@...;
release@...
Subject: Re: [genius-dev] [release] Genius CSIT failing on master
If this is master, then likely this is related to the maven issues which is creating bad artifacts so odl is in a bad state. That is being worked now.
On Wed, Apr 18, 2018, 8:51 AM Vishal Thapar <vthapar@...> wrote:
No, there hasn't been a functional change in OVSDB plugin for more than 2 releases now. There is a bug with stale connection cleanup
on OVS restart, could be related to that. Suneelu had a similar issue and fix in HWVTEP. Can check with him if this is same issue. Josh is also running into some variation of same issue and I think Sathwick was at some point trying to capture some logs for
Suneelu too.
On Wed, Apr 18, 2018 at 4:24 PM, Faseela K <faseela.k@...> wrote:
+ovsdb-dev
Vishal,
Since distribution-checks are randomly failing, don’t know whether the tests are running properly or not.
But we are hitting an issue where tunnels are not coming up on the switch, even though the tunnel information is present in config/network-topology. Is there any recent change
in ovsdb plugin?
Thanks,
Faseela
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release
_______________________________________________
genius-dev mailing list
genius-dev@...
https://lists.opendaylight.org/mailman/listinfo/genius-dev
|
|
|
