This group is locked. No changes can be made to the group while it is locked.
Date
1 - 12 of 12
OVSDB CSIT Failures on Oxygen
|
Vishal Thapar <vishal.thapar@...>
Hi Jamo,
Could you look at failures in OVSDB CSIT [0] and update Oxygen tracking sheet [1] if those are blockers or not? There seem to be 3 extra failures as compared to Nitrogen and Oxygen. Regression or newer tests failing?
Regards, Vishal.
[0] https://jenkins.opendaylight.org/releng/job/ovsdb-csit-1node-upstream-southbound-all-oxygen [1] https://docs.google.com/spreadsheets/d/1dYOY99twqHV_Q0YorAOOxmL0aFc3icNXg8qA_zGwKyA/edit#gid=727949983
|
|
Sam Hague
On Wed, Feb 28, 2018 at 7:18 AM, Vishal Thapar <vishal.thapar@...> wrote:
There are new failures. Back in December a few more test started failing. I think the vlan tagging has been there a while, but the QoS ones started failing.
|
|
Vishal Thapar <vishal.thapar@...>
Yeah, Vlan tag is the one that is there in carbon and nitrogen too. QoS ones, marked as blockers? Or target them for SR1?
Regards, Vishal.
From: Sam Hague [mailto:shague@...]
Sent: 28 February 2018 17:55 To: Vishal Thapar <vishal.thapar@...> Cc: Jamo Luhrsen <jluhrsen@...>; ovsdb-dev@... Subject: Re: [ovsdb-dev] OVSDB CSIT Failures on Oxygen
On Wed, Feb 28, 2018 at 7:18 AM, Vishal Thapar <vishal.thapar@...> wrote:
There are new failures. Back in December a few more test started failing. I think the vlan tagging has been there a while, but the QoS ones started failing.
|
|
Jamo Luhrsen <jluhrsen@...>
I have not looked at these yet. I will try to get to them asap.
toggle quoted message
Show quoted text
my guess, since I think nitro and earlier ovsdb csit is passing is that we may have blockers. JamO On 2/28/18 8:05 AM, Vishal Thapar wrote:
Yeah, Vlan tag is the one that is there in carbon and nitrogen too. QoS ones, marked as blockers? Or target them for SR1? |
|
Jamo Luhrsen <jluhrsen@...>
Vishal,
toggle quoted message
Show quoted text
here's a jira: https://jira.opendaylight.org/browse/OVSDB-453 it's marked blocker for Oxygen. How can I help to get this figured out? Thanks, JamO On 3/1/18 2:48 PM, Jamo Luhrsen wrote:
I have not looked at these yet. I will try to get to them asap. |
|
Vishal Thapar <vishal.thapar@...>
Hi Jamo,
toggle quoted message
Show quoted text
Looking at the log you pasted, I think it is related to yangtools changes that came in during Nov/Dec that added stricter checking. I'll try figure it out, maybe check with Robert on what this error means. Regards, Vishal. -----Original Message-----
From: Jamo Luhrsen [mailto:jluhrsen@...] Sent: 06 March 2018 11:17 To: Vishal Thapar <vishal.thapar@...>; Sam Hague <shague@...> Cc: ovsdb-dev@...; Kit Lou <klou.external@...> Subject: Re: [ovsdb-dev] OVSDB CSIT Failures on Oxygen Vishal, here's a jira: https://jira.opendaylight.org/browse/OVSDB-453 it's marked blocker for Oxygen. How can I help to get this figured out? Thanks, JamO On 3/1/18 2:48 PM, Jamo Luhrsen wrote: I have not looked at these yet. I will try to get to them asap. |
|
Vishal Thapar <vishal.thapar@...>
Hi Jamo,
toggle quoted message
Show quoted text
Looking at logs, noticed something else strange too in our test cases.[2] Is adding a node to 10.30.170.187:6634 with nodeId as ovsdb:HOST1. At this point we already have the same node present with nodeId 10.30.170.187:6634 in operational which was actually done in previous step [3]. So we now have two entries in operational for the same which can be seen in [4]. Is this a wrong use case or bug in code? Sam/Anil, Inputs? Do we support two connections to the same node in config? Just before the qos-entries failure I also see these log entries: 2018-03-05T07:39:38,514 | INFO | pipe-log:log "ROBOT MESSAGE: Starting test Get Operational Topology to make sure the connection has been deleted" | core | 128 - org.apache.karaf.log.core - 4.1.3 | ROBOT MESSAGE: Starting test Create OVSDB NODE HOST1 2018-03-05T07:39:38,552 | WARN | opendaylight-cluster-data-notification-dispatcher-105 | OvsdbDataTreeChangeListener | 456 - org.opendaylight.ovsdb.southbound-impl - 1.6.0.SNAPSHOT | Connection to device ConnectionInfo{getRemoteIp=IpAddress [_ipv4Address=Ipv4Address [_value=10.30.170.187]], getRemotePort=PortNumber [_value=6634], augmentations={}} already exists. Plugin does not allow multiple connections to same device, hence dropping the request OvsdbNodeAugmentation{getConnectionInfo=ConnectionInfo{getRemoteIp=IpAddress [_ipv4Address=Ipv4Address [_value=10.30.170.187]], getRemotePort=PortNumber [_value=6634], augmentations={}}} 2018-03-05T07:39:38,553 | INFO | opendaylight-cluster-data-notification-dispatcher-105 | ControllerUpdateCommand | 456 - org.opendaylight.ovsdb.southbound-impl - 1.6.0.SNAPSHOT | Register ODL controllers : {} bridges detail : {} 2018-03-05T07:39:38,632 | INFO | pipe-log:log "ROBOT MESSAGE: Starting test Get Operational Topology to make sure the connection has been deleted" | core | 128 - org.apache.karaf.log.core - 4.1.3 | ROBOT MESSAGE: Starting test Create QOS entry 2018-03-05T07:39:38,680 | INFO | opendaylight-cluster-data-notification-dispatcher-91 | ControllerUpdateCommand | 456 - org.opendaylight.ovsdb.southbound-impl - 1.6.0.SNAPSHOT | Register ODL controllers : {} bridges detail : {} 2018-03-05T07:39:38,693 | ERROR | opendaylight-cluster-data-notification-dispatcher-91 | DataTreeChangeListenerActor | 282 - org.opendaylight.controller.sal-clustering-commons - 1.7.0.SNAPSHOT | member-1-shard-topology-config: Error notifying listener org.opendaylight.ovsdb.southbound.OvsdbDataTreeChangeListener@682a9b87 java.lang.IllegalArgumentException: Failed to map QName {} [(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)qos-entries] Still looking into the yangtools issue, but could be related to this 'two entries for same in operational'. Likely it is looking for key to exist before allowing operation and is picking up the wrong/old nodeId instead of new one where we're trying to configure it. Regards, Vishal. [2] https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/ovsdb-csit-1node-upstream-southbound-all-oxygen/185/robot-plugin/log.html.gz#s1-s4-t20-k2-k6 [3] https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/ovsdb-csit-1node-upstream-southbound-all-oxygen/185/robot-plugin/log.html.gz#s1-s4-t19-k3-k2 [4] https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/ovsdb-csit-1node-upstream-southbound-all-oxygen/185/robot-plugin/log.html.gz#s1-s4 -----Original Message-----
From: Vishal Thapar Sent: 06 March 2018 11:23 To: 'Jamo Luhrsen' <jluhrsen@...>; Sam Hague <shague@...> Cc: ovsdb-dev@...; Kit Lou <klou.external@...> Subject: RE: [ovsdb-dev] OVSDB CSIT Failures on Oxygen Hi Jamo, Looking at the log you pasted, I think it is related to yangtools changes that came in during Nov/Dec that added stricter checking. I'll try figure it out, maybe check with Robert on what this error means. Regards, Vishal. -----Original Message----- From: Jamo Luhrsen [mailto:jluhrsen@...] Sent: 06 March 2018 11:17 To: Vishal Thapar <vishal.thapar@...>; Sam Hague <shague@...> Cc: ovsdb-dev@...; Kit Lou <klou.external@...> Subject: Re: [ovsdb-dev] OVSDB CSIT Failures on Oxygen Vishal, here's a jira: https://jira.opendaylight.org/browse/OVSDB-453 it's marked blocker for Oxygen. How can I help to get this figured out? Thanks, JamO On 3/1/18 2:48 PM, Jamo Luhrsen wrote: I have not looked at these yet. I will try to get to them asap. |
|
Vishal Thapar <vishal.thapar@...>
Adding yangtools and Robert [as I don't have subscription to yangtools] as this looks like old yangtools issue.
toggle quoted message
Show quoted text
Hi Yangtools devs, During yangtools migration we ran into an issue in OVSDB and OFPlugin where two versions of network-topology were being loaded which broke OVSDB's InstanceIdentifierCodec.serialize() It couldn't figure out the correct Qname when there were two revisions of same model loaded. At the time we added a work around [5] that hardcoded network-topology revision and I believe yangtools apparently fixed it. Issue with fix is that OVSDB augments network-topology and for those models revision is different. That is why I attempted a CSIT run on the patch [6] that reverts this change. But now we ran into original issue that we had put a workaround for as seen in [7], search for 'Failed to map QName {}' Any suggestions on the same? Do we need to rewrite our serializers? If yes, inputs would be welcome. [8] is the original bug raised for this. Regards, Vishal. [5] https://git.opendaylight.org/gerrit/#/c/67191/ [6] https://git.opendaylight.org/gerrit/#/c/69112/ [7] https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/ovsdb-csit-1node-gate-southbound-all-fluorine/1/odl1_karaf.log.gz [8] https://jira.opendaylight.org/browse/YANGTOOLS-844 -----Original Message-----
From: Vishal Thapar Sent: 06 March 2018 12:39 To: 'Jamo Luhrsen' <jluhrsen@...>; 'Sam Hague' <shague@...> Cc: 'ovsdb-dev@...' <ovsdb-dev@...>; 'Kit Lou' <klou.external@...> Subject: RE: [ovsdb-dev] OVSDB CSIT Failures on Oxygen Hi Jamo, Looking at logs, noticed something else strange too in our test cases.[2] Is adding a node to 10.30.170.187:6634 with nodeId as ovsdb:HOST1. At this point we already have the same node present with nodeId 10.30.170.187:6634 in operational which was actually done in previous step [3]. So we now have two entries in operational for the same which can be seen in [4]. Is this a wrong use case or bug in code? Sam/Anil, Inputs? Do we support two connections to the same node in config? Just before the qos-entries failure I also see these log entries: 2018-03-05T07:39:38,514 | INFO | pipe-log:log "ROBOT MESSAGE: Starting test Get Operational Topology to make sure the connection has been deleted" | core | 128 - org.apache.karaf.log.core - 4.1.3 | ROBOT MESSAGE: Starting test Create OVSDB NODE HOST1 2018-03-05T07:39:38,552 | WARN | opendaylight-cluster-data-notification-dispatcher-105 | OvsdbDataTreeChangeListener | 456 - org.opendaylight.ovsdb.southbound-impl - 1.6.0.SNAPSHOT | Connection to device ConnectionInfo{getRemoteIp=IpAddress [_ipv4Address=Ipv4Address [_value=10.30.170.187]], getRemotePort=PortNumber [_value=6634], augmentations={}} already exists. Plugin does not allow multiple connections to same device, hence dropping the request OvsdbNodeAugmentation{getConnectionInfo=ConnectionInfo{getRemoteIp=IpAddress [_ipv4Address=Ipv4Address [_value=10.30.170.187]], getRemotePort=PortNumber [_value=6634], augmentations={}}} 2018-03-05T07:39:38,553 | INFO | opendaylight-cluster-data-notification-dispatcher-105 | ControllerUpdateCommand | 456 - org.opendaylight.ovsdb.southbound-impl - 1.6.0.SNAPSHOT | Register ODL controllers : {} bridges detail : {} 2018-03-05T07:39:38,632 | INFO | pipe-log:log "ROBOT MESSAGE: Starting test Get Operational Topology to make sure the connection has been deleted" | core | 128 - org.apache.karaf.log.core - 4.1.3 | ROBOT MESSAGE: Starting test Create QOS entry 2018-03-05T07:39:38,680 | INFO | opendaylight-cluster-data-notification-dispatcher-91 | ControllerUpdateCommand | 456 - org.opendaylight.ovsdb.southbound-impl - 1.6.0.SNAPSHOT | Register ODL controllers : {} bridges detail : {} 2018-03-05T07:39:38,693 | ERROR | opendaylight-cluster-data-notification-dispatcher-91 | DataTreeChangeListenerActor | 282 - org.opendaylight.controller.sal-clustering-commons - 1.7.0.SNAPSHOT | member-1-shard-topology-config: Error notifying listener org.opendaylight.ovsdb.southbound.OvsdbDataTreeChangeListener@682a9b87 java.lang.IllegalArgumentException: Failed to map QName {} [(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)qos-entries] Still looking into the yangtools issue, but could be related to this 'two entries for same in operational'. Likely it is looking for key to exist before allowing operation and is picking up the wrong/old nodeId instead of new one where we're trying to configure it. Regards, Vishal. [2] https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/ovsdb-csit-1node-upstream-southbound-all-oxygen/185/robot-plugin/log.html.gz#s1-s4-t20-k2-k6 [3] https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/ovsdb-csit-1node-upstream-southbound-all-oxygen/185/robot-plugin/log.html.gz#s1-s4-t19-k3-k2 [4] https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/ovsdb-csit-1node-upstream-southbound-all-oxygen/185/robot-plugin/log.html.gz#s1-s4 -----Original Message----- From: Vishal Thapar Sent: 06 March 2018 11:23 To: 'Jamo Luhrsen' <jluhrsen@...>; Sam Hague <shague@...> Cc: ovsdb-dev@...; Kit Lou <klou.external@...> Subject: RE: [ovsdb-dev] OVSDB CSIT Failures on Oxygen Hi Jamo, Looking at the log you pasted, I think it is related to yangtools changes that came in during Nov/Dec that added stricter checking. I'll try figure it out, maybe check with Robert on what this error means. Regards, Vishal. -----Original Message----- From: Jamo Luhrsen [mailto:jluhrsen@...] Sent: 06 March 2018 11:17 To: Vishal Thapar <vishal.thapar@...>; Sam Hague <shague@...> Cc: ovsdb-dev@...; Kit Lou <klou.external@...> Subject: Re: [ovsdb-dev] OVSDB CSIT Failures on Oxygen Vishal, here's a jira: https://jira.opendaylight.org/browse/OVSDB-453 it's marked blocker for Oxygen. How can I help to get this figured out? Thanks, JamO On 3/1/18 2:48 PM, Jamo Luhrsen wrote: I have not looked at these yet. I will try to get to them asap. |
|
Robert Varga <nite@...>
On 06/03/18 10:34, Vishal Thapar wrote:
Adding yangtools and Robert [as I don't have subscription to yangtools] as this looks like old yangtools issue.The original issue was a ordering one, now it would seem you cannot find the module in the SchemaContext -- at this point the question is, what modules are in the SchemaContext -- you'll need to debug that on your side... Regards, Robert |
|
Robert Varga <nite@...>
On 06/03/18 11:15, Robert Varga wrote:
More specifically, it seems your class is not thread-safe w.r.t.Any suggestions on the same? Do we need to rewrite our serializers? If yes, inputs would be welcome. [8] is the original bug raised for this.The original issue was a ordering one, now it would seem you cannot find SchemaContext updates. Bye, Robert |
|
Vishal Thapar <vishal.thapar@...>
Stephen,
toggle quoted message
Show quoted text
Any comments on thread safety of code calling serialize? Note that these failures are on a patch that reverted the workaround that we did for yangtools issue. There are no other changes in OVSDB code, except the checkstyle etc. fixes. Regards, Vishal. -----Original Message-----
From: Robert Varga [mailto:nite@...] Sent: 06 March 2018 16:19 To: Vishal Thapar <vishal.thapar@...>; Jamo Luhrsen <jluhrsen@...>; Sam Hague <shague@...> Cc: ovsdb-dev@...; Kit Lou <klou.external@...>; yangtools-dev <yangtools-dev@...> Subject: Re: [ovsdb-dev] OVSDB CSIT Failures on Oxygen On 06/03/18 11:15, Robert Varga wrote: More specifically, it seems your class is not thread-safe w.r.t.Any suggestions on the same? Do we need to rewrite our serializers? If yes, inputs would be welcome. [8] is the original bug raised for this.The original issue was a ordering one, now it would seem you cannot SchemaContext updates. Bye, Robert |
|
Sam Hague
Vishal, Tom, can you look at the comment from Robert on the original patch that set the codec iid and see if that moves us forward? Thanks, Sam On Thu, Mar 8, 2018 at 1:03 AM, Vishal Thapar <vishal.thapar@...> wrote: Stephen, |