|
Re: [release] controller CSIT for Aluminium
Robert Varga
On 09/06/2020 02:49, JamO Luhrsen wrote:
Hi all,[snip] Trouble on the test side:This is across all releases, I just updated the ticket and raised it to blocker, as it does block any attempt to release anything. I have attached known-good and known-bad console.log and the 'diff -U 0' between them. It seems to be pointing at https://git.opendaylight.org/gerrit/c/releng/builder/+/90196/2/jjb/integration/integration-templates.yaml which would seem to indicate trouble with python setup in the new image. Thanh, any idea on how to debug this further? Thanks, Robert |
|
|
|
Re: validate-autorelease-aluminium failing
Luis Gomez
The check conditions have changed after we introduced MRI projects, so now it is possible to have projects in distribution but not in AR. Because of this, we can either rework the script to check something different (not sure what) or just remove as I did.
toggle quoted message
Show quoted text
BR/Luis
|
|
|
|
Re: validate-autorelease-aluminium failing
Thanh ha <zxiiro@...>
This just turns off the comparison entirely so not really fixing but I think it's the right approach until someone can fix the script. Did a quick look at the script and it looks like it parses pom.xml files dynamically to figure out what to compare which means we definitely need to adjust the script to allow exceptions for MRI projects. Regards, Thanh On Fri, Jun 12, 2020 at 12:22 PM Luis Gomez <ecelgp@...> wrote:
|
|
|
|
Re: validate-autorelease-aluminium failing
Luis Gomez
toggle quoted message
Show quoted text
|
|
|
|
Re: validate-autorelease-aluminium failing
Robert Varga
Well, we fixed feature packaging in distribution (i.e. we now include
toggle quoted message
Show quoted text
features-controller-testing and others). Also, validate-autorelease triggers only when pom.xmls are modified I think, hence it is not typically seen. Regards, Robert On 12/06/2020 18:11, Luis Gomez wrote:
I think you are right, but why this starts to fail now?On Jun 11, 2020, at 11:41 PM, Robert Varga <nite@...> wrote: |
|
|
|
Re: validate-autorelease-aluminium failing
Luis Gomez
I think you are right, but why this starts to fail now?
toggle quoted message
Show quoted text
On Jun 11, 2020, at 11:41 PM, Robert Varga <nite@...> wrote: |
|
|
|
validate-autorelease-aluminium failing
Robert Varga
Hello,
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/infrautils-validate-autorelease-aluminium/2/console.log.gz failed with: [infrautils-validate-autorelease-aluminium] $ /bin/bash -l /tmp/jenkins5280574077411919720.shI think this is wrong, since MRI projects need to be in the distribution and not be present in autorelease. Perhaps compare-autorelease-projects.sh needs to be updated with list of MRI projects? Regards, Robert |
|
|
|
integration meeting minutes (6/10/2020)
JamO Luhrsen
|
|
|
|
integration/test verify tox job failing.
JamO Luhrsen
I've been trying to figure it out since this afternoon. Here's a jira for it for now:
https://jira.opendaylight.org/browse/RELENG-149 JamO |
|
|
|
integration meeting tomorrow (6/10/2020)
JamO Luhrsen
Just a reminder that we will hold the call tomorrow 9pm PST. At least
we can discuss the current state of Aluminium and status of recent issues filed: https://lists.opendaylight.org/g/integration-dev/message/14173 Also, another thing to discuss is that stable/sodium is now locked, but there are some useful docs patches on that branch we should probably merge: https://git.opendaylight.org/gerrit/q/project:docs+branch:stable/sodium+status:open Bring other topics as needed. Thanks, JamO |
|
|
|
controller CSIT for Aluminium
JamO Luhrsen
Hi all,
I've had an action item for a while to dig through Aluminium controller CSIT to try to get a handle on where we stand after the MRI bumps. This has turned up some jira tickets: kernel projects issues to dig deeper on: https://jira.opendaylight.org/browse/CONTROLLER-1944 https://jira.opendaylight.org/browse/MDSAL-568 https://jira.opendaylight.org/browse/CONTROLLER-1945 https://jira.opendaylight.org/browse/CONTROLLER-1946 Trouble on the test side: https://jira.opendaylight.org/browse/INTTEST-99 https://jira.opendaylight.org/browse/INTTEST-100 https://jira.opendaylight.org/browse/INTTEST-101 https://jira.opendaylight.org/browse/INTTEST-102 Anyone that has some time is welcome to look at all of the above and create gerrits and/or add more details to the tickets. Once we get past these, we can look again. The hope is that fixing things at the controller project level first will give us improvements in other projects before we really dig in to those. Thanks, JamO #aluminium #csit |
|
|
|
Re: Upgrade script - CSIT
Dheenadayalan B <dheenadayalan.b@...>
Hi Srini/Karthik/Devteam, Unlike in magnesium branch, northbound interface addition leads to time delay of more than one minute, leads to upgrade csit failure, Openflowplugin module was listening to port 6653/6633 , however connection establishment between ovs and plugin made with a delay of one minute through northbound.
logs as below: Openflowplugin startup completed as below:
Karaf.log: 020-06-04T02:28:15,339 | INFO | SystemReadyService-0 | OpenFlowPluginProviderImpl | 382 - org.opendaylight.openflowplugin.impl - 0.11.0.SNAPSHOT | onSystemBootReady() received, starting the switch connections 2020-06-04T02:28:15,547 | INFO | OFP-SwitchConnectionProvider-Udp/TcpHandler_6653-0 | TcpHandler | 392 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.11.0.SNAPSHOT | Switch listener started and ready to accept incoming tcp/tls connections on port: 6653 2020-06-04T02:28:15,550 | INFO | OFP-SwitchConnectionProvider-Udp/TcpHandler_6653-0 | OpenflowDiagStatusProviderImpl | 392 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.11.0.SNAPSHOT | The report status is OPERATIONAL for OPENFLOW_SERVER_6653 2020-06-04T02:28:15,559 | INFO | SystemReadyService-0 | SouthboundProvider | 401 - org.opendaylight.ovsdb.southbound-impl - 1.11.0.SNAPSHOT | Started OVSDB Manager (in system ready listener) 2020-06-04T02:28:15,561 | INFO | OFP-SwitchConnectionProvider-Udp/TcpHandler_6633-0 | OpenFlowPluginProviderImpl | 382 - org.opendaylight.openflowplugin.impl - 0.11.0.SNAPSHOT | All switchConnectionProviders are up and running (2). 2020-06-04T02:28:15,561 | INFO | OFP-SwitchConnectionProvider-Udp/TcpHandler_6633-0 | OpenflowDiagStatusProviderImpl | 392 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.11.0.SNAPSHOT | The report status is OPERATIONAL for OPENFLOW
Script for northbound setup executed @20200604 07:58:43.060 (UTC+05:30) (2020-06-04T02:28:43)
sudo ovs-vsctl set-manager tcp:10.30.170.113:6640 ptcp:6641:127.0.0.1
karaf.log:
2020-06-04T02:28:43,942 | DEBUG | jobcoordinator-main-task-6 | InterfaceTopologyStateListener | 248 - org.opendaylight.genius.interfacemanager-impl - 0.9.0.SNAPSHOT | adding bridge references for bridge: OvsdbBridgeAugmentation{getBridgeExternalIds={BridgeExternalIdsKey{_bridgeExternalIdKey=created_by}=BridgeExternalIds{getBridgeExternalIdKey=created_by, getBridgeExternalIdValue=odl, augmentation=[]}, BridgeExternalIdsKey{_bridgeExternalIdKey=opendaylight-iid}=BridgeExternalIds{getBridgeExternalIdKey=opendaylight-iid, getBridgeExternalIdValue=/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/758da02d-7c35-43b6-99aa-c9c57108210f/bridge/br-int'], augmentation=[]}}, getBridgeName=OvsdbBridgeName{_value=br-int}, getBridgeOtherConfigs={BridgeOtherConfigsKey{_bridgeOtherConfigKey=hwaddr}=BridgeOtherConfigs{getBridgeOtherConfigKey=hwaddr, getBridgeOtherConfigValue=ec:47:56:1c:b1:9b, augmentation=[]}, BridgeOtherConfigsKey{_bridgeOtherConfigKey=disable-in-band}=BridgeOtherConfigs{getBridgeOtherConfigKey=disable-in-band, getBridgeOtherConfigValue=true, augmentation=[]}}, getBridgeUuid=Uuid{_value=935221ad-40ad-4aaa-939c-a8654e883bd8}, getDatapathId=DatapathId{_value=00:00:ec:47:56:1c:b1:9b}, getDatapathType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.DatapathTypeSystem, getFailMode=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbFailModeSecure, getManagedBy=OvsdbNodeRef{_value=KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology, org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey{_topologyId=Uri{_value=ovsdb:1}}], org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey{_nodeId=Uri{_value=ovsdb://uuid/758da02d-7c35-43b6-99aa-c9c57108210f}}]]}}, getProtocolEntry={ProtocolEntryKey{_protocol=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbBridgeProtocolOpenflow13}=ProtocolEntry{getProtocol=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbBridgeProtocolOpenflow13, augmentation=[]}}, isStpEnable=false}, dpn: 259791131554203
Openflowplugin and ovs communication time: Karaf.log:
2020-06-04T02:29:56,797 | INFO | epollEventLoopGroup-7-3 | ContextChainHolderImpl | 382 - org.opendaylight.openflowplugin.impl - 0.11.0.SNAPSHOT | Device openflow:259791131554203 connected.
Ovs-vswitchd.log: 2020-06-04T02:29:56.768Z|01001|rconn|INFO|br-int<->tcp:10.30.170.113:6653: connected
Thanks, Dheena
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Sent: 05 June 2020 11:31 To: 'Dheenadayalan B' <dheenadayalan.b@...>; 'odl netvirt dev' <netvirt-dev@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; karthikeyan.k@...; 'Chetan Arakere Gowdru' <chetan.arakere@...>; integration-dev@... Subject: RE: Upgrade script - CSIT
++ Netvirt-Dev
Thanks, Srinivas +91-9243478719
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Hi Dheena,
Thanks for the debug session we had today.
The issue here is the bundle came up around 1min 58sec, while my script looks for 1min.
Now the question still remains if we need to wait more time for bundle to come up in Aluminium, while for Magnesium it came up within 1 min.
So today’s action items are:
Below is bundle URL. Magnesium:
Magnesium ODL:
Thanks, Srinivas +91-9243478719
From: Dheenadayalan B <dheenadayalan.b@...>
Hi Srini, RPC will be registered on openflowdevice(DPN) connected with the controller. From the logs provided observerved rpc call executed even before dpn 259791131554203 connection established, hence the error.
Rpc call execution time: 20200604 07:58:56.147 / 20200604 07:58:56.160 / 00:00:00.013 (2020-06-04T02:28:56) (UTC+05:30hrs)
body={ "input": { "node": "/opendaylight-inventory:nodes/opendaylight-inventory:node[opendaylight-inventory:id='openflow:259791131554203']", "node-id": 259791131554203 }
CSIT logs: (UTC time zone)
2020-06-04T02:29:56,797 | INFO | epollEventLoopGroup-7-3 | ContextChainHolderImpl | 382 - org.opendaylight.openflowplugin.impl - 0.11.0.SNAPSHOT | Device openflow:259791131554203 connected.
Could you please delay script execution by one minute and check if the same issue occurring again?
Thanks, Dheena
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
https://jira.opendaylight.org/browse/NETVIRT-1672
Thanks, Srinivas +91-9243478719
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Please see the below error:
POST Response : url=http://10.30.170.113:8181/restconf/operations/arbitrator-reconcile:get-active-bundle status=501, reason=Not Implemented
Can this be added to Aluminium. We have aluminium release in next 15 days and this fix is needed.
Thanks, Srinivas +91-9243478719
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Hi Dheena,
The fixes passed in Magnesium but failed in Aluminium.
Please kindly look into this on priority and let me know if the below json needs to be changed for Aluminium.
Thanks, Srinivas +91-9243478719
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Hi Dheena,
Thanks a lot for the help. With your suggestions I was able to run the scripts and it went fine.
Can you please mail me in details the reasons for the changes so that I can ask the community for acceptance.
Thanks, Srinivas +91-9243478719
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Hi Dheena,
I am working on reproducing the upgrade issue in CSIT today. I will ping once I am at failed state.
We will debug it together on my local setup.
Thanks, Srinivas +91-9243478719
|
|
|
|
Re: Exception during netvirt CSIT - Blocking Aluminium
Srinivas <srinivas.rachakonda@...>
Hi,
I have opened the below JIRA.
https://jira.opendaylight.org/browse/NETVIRT-1671
Thanks, Srinivas +91-9243478719
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Sent: 08 June 2020 09:26 To: 'karthikeyan.k@...' <karthikeyan.k@...>; 'Nishchya Gupta' <nishchyag@...>; 'Somashekar Byrappa' <somashekar.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...>; 'Dilliraj Kuppusamy' <dilliraj.k@...>; 'Dheenadayalan B' <dheenadayalan.b@...> Subject: RE: Exception during netvirt CSIT - Blocking Aluminium
Hi Karthikeyan,
The jobs are still failing with below exceptions: (This is still blocking Aluminium)
Karaf logs:
| Job still failed on final retry: JobEntry{key='bc.group.update.376b8a1f-a6a4-4872-8dc1-5303b57267ed', jobId='J377', queueId='Q274', mainWorker=org.opendaylight.netvirt.elan.l2gw.jobs.BcGroupUpdateJob@62da31a7, rollbackWorker=null, retryCount=5/5, futures=[com.google.common.util.concurrent.ImmediateFuture$ImmediateFailedFuture@91b9f64[status=FAILURE, cause=[java.lang.IllegalArgumentException: Multiple entries with same key: BucketKey{_bucketId=BucketId{_value=1}}=Bucket{_action={ActionKey{_order=0}=Action{_action=NxActionRegLoadNodesNodeTableFlowApplyActionsCase{_nxRegLoad=NxRegLoad{_dst=Dst{_dstChoice=DstNxRegCase{_nxReg=interface org.opendaylight.yang.gen.v1.urn.opendaylight.openflowjava.nx.match.rev140421.NxmNxReg6, augmentation=[]}, _end=31, _start=0, augmentation=[]}, _value=3328, augmentation=[]}, augmentation=[]}, _order=0, augmentation=[]}, ActionKey{_order=1}=Action{_action=NxActionResubmitRpcAddGroupCase{_nxResubmit=NxResubmit{_table=220, augmentation=[]}, augmentation=[]}, _order=1, augmentation=[]}}, _bucketId=BucketId{_value=1}, _watchGroup=4294967295, _watchPort=4294967295, _weight=0, augmentation=[]} and BucketKey{_bucketId=BucketId{_value=1}}=Bucket{_action={ActionKey{_order=0}=Action{getAction=NxActionRegLoadNodesNodeGroupBucketsBucketActionsCase{getNxRegLoad=NxRegLoad{getDst=Dst{getDstChoice=DstNxRegCase{getNxReg=interface org.opendaylight.yang.gen.v1.urn.opendaylight.openflowjava.nx.match.rev140421.NxmNxReg6, augmentation=[]}, getEnd=31, getStart=0, augmentation=[]}, getValue=3328, augmentation=[]}, augmentation=[]}, getOrder=0, augmentation=[]}, ActionKey{_order=1}=Action{getAction=NxActionResubmitNodesNodeGroupBucketsBucketActionsCase{getNxResubmit=NxResubmit{getInPort=65528, getTable=220, augmentation=[]}, augmentation=[]}, getOrder=1, augmentation=[]}}, _bucketId=BucketId{_value=1}, _watchGroup=4294967295, _watchPort=4294967295, _weight=0, augmentation=[]}. To index multiple values under a key, use Multimaps.index.]]]} java.lang.IllegalArgumentException: Multiple entries with same key: BucketKey{_bucketId=BucketId{_value=1}}=Bucket{_action={ActionKey{_order=0}=Action{_action=NxActionRegLoadNodesNodeTableFlowApplyActionsCase{_nxRegLoad=NxRegLoad{_dst=Dst{_dstChoice=DstNxRegCase{_nxReg=interface org.opendaylight.yang.gen.v1.urn.opendaylight.openflowjava.nx.match.rev140421.NxmNxReg6, augmentation=[]}, _end=31, _start=0, augmentation=[]}, _value=3328, augmentation=[]}, augmentation=[]}, _order=0, augmentation=[]}, ActionKey{_order=1}=Action{_action=NxActionResubmitRpcAddGroupCase{_nxResubmit=NxResubmit{_table=220, augmentation=[]}, augmentation=[]}, _order=1, augmentation=[]}}, _bucketId=BucketId{_value=1}, _watchGroup=4294967295, _watchPort=4294967295, _weight=0, augmentation=[]} and BucketKey{_bucketId=BucketId{_value=1}}=Bucket{_action={ActionKey{_order=0}=Action{getAction=NxActionRegLoadNodesNodeGroupBucketsBucketActionsCase{getNxRegLoad=NxRegLoad{getDst=Dst{getDstChoice=DstNxRegCase{getNxReg=interface org.opendaylight.yang.gen.v1.urn.opendaylight.openflowjava.nx.match.rev140421.NxmNxReg6, augmentation=[]}, getEnd=31, getStart=0, augmentation=[]}, getValue=3328, augmentation=[]}, augmentation=[]}, getOrder=0, augmentation=[]}, ActionKey{_order=1}=Action{getAction=NxActionResubmitNodesNodeGroupBucketsBucketActionsCase{getNxResubmit=NxResubmit{getInPort=65528, getTable=220, augmentation=[]}, augmentation=[]}, getOrder=1, augmentation=[]}}, _bucketId=BucketId{_value=1}, _watchGroup=4294967295, _watchPort=4294967295, _weight=0, augmentation=[]}. To index multiple values under a key, use Multimaps.index. at com.google.common.collect.Maps.uniqueIndex(Maps.java:1338) ~[bundleFile:?] at com.google.common.collect.Maps.uniqueIndex(Maps.java:1293) ~[bundleFile:?] at org.opendaylight.yangtools.yang.binding.CodeHelpers.compatMap(CodeHelpers.java:296) ~[?:?] at org.opendaylight.yang.gen.v1.urn.opendaylight.group.types.rev131018.group.BucketsBuilder.setBucket(BucketsBuilder.java:113) ~[?:?] at org.opendaylight.genius.mdsalutil.MDSALUtil.buildBucketLists(MDSALUtil.java:293) ~[?:?] at org.opendaylight.netvirt.elan.utils.ElanUtils.syncUpdateGroup(ElanUtils.java:1768) ~[?:?] at org.opendaylight.netvirt.elan.l2gw.utils.ElanL2GatewayMulticastUtils.setupStandardElanBroadcastGroups(ElanL2GatewayMulticastUtils.java:303) ~[?:?] at org.opendaylight.netvirt.elan.l2gw.utils.ElanL2GatewayMulticastUtils.updateRemoteBroadcastGroupForAllElanDpns(ElanL2GatewayMulticastUtils.java:265) ~[?:?] at org.opendaylight.netvirt.elan.l2gw.jobs.BcGroupUpdateJob.lambda$call$0(BcGroupUpdateJob.java:59) ~[?:?] at org.opendaylight.genius.infra.ManagedTransactionFactoryImpl.lambda$callWithNewTransactionAndSubmit$3(ManagedTransactionFactoryImpl.java:89) ~[?:?] at org.opendaylight.genius.infra.ManagedTransactionFactoryImpl.applyWithNewTransactionAndSubmit(ManagedTransactionFactoryImpl.java:105) ~[?:?] at org.opendaylight.genius.infra.ManagedTransactionFactoryImpl.callWithNewTransactionAndSubmit(ManagedTransactionFactoryImpl.java:88) ~[?:?] at org.opendaylight.genius.infra.ManagedNewTransactionRunnerImpl.callWithNewWriteOnlyTransactionAndSubmit(ManagedNewTransactionRunnerImpl.java:63) ~[?:?] at org.opendaylight.netvirt.elan.l2gw.jobs.BcGroupUpdateJob.call(BcGroupUpdateJob.java:58) ~[?:?] at org.opendaylight.netvirt.elan.l2gw.jobs.BcGroupUpdateJob.call(BcGroupUpdateJob.java:27) ~[?:?] at org.opendaylight.infrautils.jobcoordinator.internal.JobCoordinatorImpl$MainTask.run(JobCoordinatorImpl.java:416) [bundleFile:?] at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426) [?:?] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) [?:?] at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) [?:?] at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) [?:?] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) [?:?] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) [?:?]
Thanks, Srinivas +91-9243478719
From: karthikeyan.k@... <karthikeyan.k@...>
Thanks Nishchya for the code review.
@'srinivas.rachakonda@...', Fix got merged now.
Thanks & Regards, Karthikeyan.
From: karthikeyan.k@... <karthikeyan.k@...>
Hi Nishchya,
Raised below review to address the reported exception.
https://git.opendaylight.org/gerrit/c/netvirt/+/90296 (@Som/Nishchya, Please review and provide your comments if you have any better approach to address it)
Thanks & Regards, Karthikeyan. From: Nishchya Gupta <nishchyag@...>
Hi Kartik/Som,
Below mentioned exception is seen, during netvirt CSIT multiple times. This got introduced during MRI, List to Map changes. Where same key is used, to add multiple entries, which was not a issue while using List.
Please look into it and suggest the way to overcome.
Regards, Nishchya
2020-06-03T14:13:55,986 | INFO | jobcoordinator-main-task-139 | SubnetOpDpnManager | 364 - org.opendaylight.netvirt.vpnmanager-impl - 0.11.0.SNAPSHOT | addPortOpDataEntry: Created PortOpData entry for port 6e8a7302-66fb-43f7-a5ec-eab4497fe1df with DPNId 9628305181018 subnetId 1b022b1c-5c20-465c-90c6-dce2518a054f 2020-06-03T14:13:55,986 | ERROR | jobcoordinator-main-task-139 | VpnSubnetRouteHandler | 364 - org.opendaylight.netvirt.vpnmanager-impl - 0.11.0.SNAPSHOT | SUBNETROUTE: onInterfaceUp: Unable to handle interface up event for port 6e8a7302-66fb-43f7-a5ec-eab4497fe1df in subnet 1b022b1c-5c20-465c-90c6-dce2518a054f java.lang.IllegalArgumentException: Multiple entries with same key: VpnInterfacesKey{_interfaceName=6e8a7302-66fb-43f7-a5ec-eab4497fe1df}=VpnInterfaces{_interfaceName=6e8a7302-66fb-43f7-a5ec-eab4497fe1df, augmentation=[]} and VpnInterfacesKey{_interfaceName=6e8a7302-66fb-43f7-a5ec-eab4497fe1df}=VpnInterfaces{getInterfaceName=6e8a7302-66fb-43f7-a5ec-eab4497fe1df, augmentation=[]}. To index multiple values under a key, use Multimaps.index. at com.google.common.collect.Maps.uniqueIndex(Maps.java:1338) ~[bundleFile:?] at com.google.common.collect.Maps.uniqueIndex(Maps.java:1293) ~[bundleFile:?] at org.opendaylight.yangtools.yang.binding.CodeHelpers.compatMap(CodeHelpers.java:296) ~[bundleFile:?] at org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.l3vpn.rev130911.subnet.op.data.subnet.op.data.entry.SubnetToDpnBuilder.setVpnInterfaces(SubnetToDpnBuilder.java:148) ~[bundleFile:?] at org.opendaylight.netvirt.vpnmanager.SubnetOpDpnManager.addInterfaceToDpn(SubnetOpDpnManager.java:137) ~[bundleFile:?] at org.opendaylight.netvirt.vpnmanager.VpnSubnetRouteHandler.onInterfaceUp(VpnSubnetRouteHandler.java:613) [bundleFile:?] at org.opendaylight.netvirt.vpnmanager.SubnetRouteInterfaceStateChangeListener.lambda$update$2(SubnetRouteInterfaceStateChangeListener.java:254) [bundleFile:?] at org.opendaylight.infrautils.jobcoordinator.internal.JobCoordinatorImpl$MainTask.run(JobCoordinatorImpl.java:416) [bundleFile:?] at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426) [?:?] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) [?:?] at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) [?:?] at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) [?:?] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) [?:?] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) [?:?] 2020-06-03T14:13:55,993 | WARN | ElanOvsdbNodeListener-0 | SouthboundUtils | 405 - org.opendaylight.ovsdb.utils.southbound-utils - 1.11.0.SNAPSHOT | Ovsdb Node does not contain connection info: Node{getNodeId=Uri{_value=ovsdb://uuid/dbcd87b1-5ec5-4207-82b6-5b44fe90a41a}, augmentation=[OvsdbNodeAugmentation{getConnectionInfo=ConnectionInfo{getLocalIp=IpAddress{_ipv4Address=Ipv4Address{_value=10.30.170.9}}, getLocalPort=PortNumber{_value=6640}, getRemoteIp=IpAddress{_ipv4Address=Ipv4Address{_value=10.30.170.29}}, getRemotePort=PortNumber{_value=33446}, augmentation=[]}, getDatapathTypeEntry={DatapathTypeEntryKey{_datapathType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.DatapathTypeSystem}=DatapathTypeEntry{getDatapathType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.DatapathTypeSystem, augmentation=[]}, DatapathTypeEntryKey{_datapathType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.DatapathTypeNetdev}=DatapathTypeEntry{getDatapathType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.DatapathTypeNetdev, augmentation=[]}}, getDbVersion=7.15.0, getInterfaceTypeEntry={InterfaceTypeEntryKey{_interfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeLisp}=InterfaceTypeEntry{getInterfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeLisp, augmentation=[]}, InterfaceTypeEntryKey{_interfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeGre}=InterfaceTypeEntry{getInterfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeGre, augmentation=[]}, InterfaceTypeEntryKey{_interfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeGeneve}=InterfaceTypeEntry{getInterfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeGeneve, augmentation=[]}, InterfaceTypeEntryKey{_interfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeSystem}=InterfaceTypeEntry{getInterfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeSystem, augmentation=[]}, InterfaceTypeEntryKey{_interfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeVxlan}=InterfaceTypeEntry{getInterfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeVxlan, augmentation=[]}, InterfaceTypeEntryKey{_interfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeStt}=InterfaceTypeEntry{getInterfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeStt, augmentation=[]}, InterfaceTypeEntryKey{_interfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeTap}=InterfaceTypeEntry{getInterfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeTap, augmentation=[]}, InterfaceTypeEntryKey{_interfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypePatch}=InterfaceTypeEntry{getInterfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypePatch, augmentation=[]}, InterfaceTypeEntryKey{_interfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeInternal}=InterfaceTypeEntry{getInterfaceType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeInternal, augmentation=[]}}, getManagedNodeEntry={ManagedNodeEntryKey{_bridgeRef=OvsdbBridgeRef{_value=KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology, org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey{_topologyId=Uri{_value=ovsdb:1}}], org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey{_nodeId=Uri{_value=ovsdb://uuid/dbcd87b1-5ec5-4207-82b6-5b44fe90a41a/bridge/br-physnet1}}]]}}}=ManagedNodeEntry{getBridgeRef=OvsdbBridgeRef{_value=KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology, org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey{_topologyId=Uri{_value=ovsdb:1}}], org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey{_nodeId=Uri{_value=ovsdb://uuid/dbcd87b1-5ec5-4207-82b6-5b44fe90a41a/bridge/br-physnet1}}]]}}, augmentation=[]}, ManagedNodeEntryKey{_bridgeRef=OvsdbBridgeRef{_value=KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology, org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey{_topologyId=Uri{_value=ovsdb:1}}], org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey{_nodeId=Uri{_value=ovsdb://uuid/dbcd87b1-5ec5-4207-82b6-5b44fe90a41a/bridge/br-int}}]]}}}=ManagedNodeEntry{getBridgeRef=OvsdbBridgeRef{_value=KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology, org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey{_topologyId=Uri{_value=ovsdb:1}}], org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey{_nodeId=Uri{_value=ovsdb://uuid/dbcd87b1-5ec5-4207-82b6-5b44fe90a41a/bridge/br-int}}]]}}, augmentation=[]}}, getManagerEntry={ManagerEntryKey{_target=Uri{_value=tcp:10.30.170.9:6640}}=ManagerEntry{getNumberOfConnections=1, getTarget=Uri{_value=tcp:10.30.170.9:6640}, isConnected=true, augmentation=[]}, ManagerEntryKey{_target=Uri{_value=ptcp:6641:127.0.0.1}}=ManagerEntry{getNumberOfConnections=1, getTarget=Uri{_value=ptcp:6641:127.0.0.1}, isConnected=true, augmentation=[]}}, getOpenvswitchExternalIds={OpenvswitchExternalIdsKey{_externalIdKey=system-id}=OpenvswitchExternalIds{getExternalIdKey=system-id, getExternalIdValue=744571f3-e968-41b7-9b3a-e41c17f54f3f, augmentation=[]}, OpenvswitchExternalIdsKey{_externalIdKey=odl_os_hostconfig_hostid}=OpenvswitchExternalIds{getExternalIdKey=odl_os_hostconfig_hostid, getExternalIdValue=releng-07016-60-1-devstack-queens-0.vexxhost.local, augmentation=[]}, OpenvswitchExternalIdsKey{_externalIdKey=rundir}=OpenvswitchExternalIds{getExternalIdKey=rundir, getExternalIdValue=/var/run/openvswitch, augmentation=[]}, OpenvswitchExternalIdsKey{_externalIdKey=odl_os_hostconfig_config_odl_l2}=OpenvswitchExternalIds{getExternalIdKey=odl_os_hostconfig_config_odl_l2, getExternalIdValue={"allowed_network_types": ["local", "flat", "vlan", "vxlan", "gre"], "bridge_mappings": {"physnet1": "br-physnet1"}, "datapath_type": "system", "supported_vnic_types": [{"vif_type": "ovs", "vnic_type": "normal", "vif_details": {"support_vhost_user": false, "has_datapath_type_netdev": false, "uuid": "dbcd87b1-5ec5-4207-82b6-5b44fe90a41a", "host_addresses": ["releng-07016-60-1-devstack-queens-0.vexxhost.local"]}}]}, augmentation=[]}, OpenvswitchExternalIdsKey{_externalIdKey=hostname}=OpenvswitchExternalIds{getExternalIdKey=hostname, getExternalIdValue=localhost, augmentation=[]}}, getOpenvswitchOtherConfigs={OpenvswitchOtherConfigsKey{_otherConfigKey=local_ip}=OpenvswitchOtherConfigs{getOtherConfigKey=local_ip, getOtherConfigValue=10.30.170.29, augmentation=[]}, OpenvswitchOtherConfigsKey{_otherConfigKey=provider_mappings}=OpenvswitchOtherConfigs{getOtherConfigKey=provider_mappings, getOtherConfigValue=physnet1:br-physnet1, augmentation=[]}}, getOvsVersion=2.8.2}]} |
|
|
|
Re: Upgrade script - CSIT
Srinivas <srinivas.rachakonda@...>
++ Netvirt-Dev
Thanks, Srinivas +91-9243478719
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Sent: 05 June 2020 11:13 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'karthikeyan.k@...' <karthikeyan.k@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT
Hi Dheena,
Thanks for the debug session we had today.
The issue here is the bundle came up around 1min 58sec, while my script looks for 1min.
Now the question still remains if we need to wait more time for bundle to come up in Aluminium, while for Magnesium it came up within 1 min.
So today’s action items are:
Below is bundle URL. Magnesium:
Magnesium ODL:
Thanks, Srinivas +91-9243478719
From: Dheenadayalan B <dheenadayalan.b@...>
Hi Srini, RPC will be registered on openflowdevice(DPN) connected with the controller. From the logs provided observerved rpc call executed even before dpn 259791131554203 connection established, hence the error.
Rpc call execution time: 20200604 07:58:56.147 / 20200604 07:58:56.160 / 00:00:00.013 (2020-06-04T02:28:56) (UTC+05:30hrs)
body={ "input": { "node": "/opendaylight-inventory:nodes/opendaylight-inventory:node[opendaylight-inventory:id='openflow:259791131554203']", "node-id": 259791131554203 }
CSIT logs: (UTC time zone)
2020-06-04T02:29:56,797 | INFO | epollEventLoopGroup-7-3 | ContextChainHolderImpl | 382 - org.opendaylight.openflowplugin.impl - 0.11.0.SNAPSHOT | Device openflow:259791131554203 connected.
Could you please delay script execution by one minute and check if the same issue occurring again?
Thanks, Dheena
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
https://jira.opendaylight.org/browse/NETVIRT-1672
Thanks, Srinivas +91-9243478719
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Please see the below error:
POST Response : url=http://10.30.170.113:8181/restconf/operations/arbitrator-reconcile:get-active-bundle status=501, reason=Not Implemented
Can this be added to Aluminium. We have aluminium release in next 15 days and this fix is needed.
Thanks, Srinivas +91-9243478719
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Hi Dheena,
The fixes passed in Magnesium but failed in Aluminium.
Please kindly look into this on priority and let me know if the below json needs to be changed for Aluminium.
Thanks, Srinivas +91-9243478719
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Hi Dheena,
Thanks a lot for the help. With your suggestions I was able to run the scripts and it went fine.
Can you please mail me in details the reasons for the changes so that I can ask the community for acceptance.
Thanks, Srinivas +91-9243478719
From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Hi Dheena,
I am working on reproducing the upgrade issue in CSIT today. I will ping once I am at failed state.
We will debug it together on my local setup.
Thanks, Srinivas +91-9243478719
|
|
|
|
Re: integration meeting minutes (5/27/2020)
Srinivas <srinivas.rachakonda@...>
Hi Anil,
Attached is the excel sheet for netvirt jobs.
Tempest jobs are not added in this sheet.
Thanks, Srinivas +91-9243478719
From: Jamo Luhrsen <jluhrsen@...>
Sent: 04 June 2020 08:43 To: Anil Belur <abelur@...>; srinivas.rachakonda@...; srinivas.rachakonda@... Cc: integration-dev@... Subject: Re: [integration-dev] integration meeting minutes (5/27/2020)
On 6/3/20 7:21 PM, Anil Belur wrote:
Anil, the list is there in the meeting minutes: https://wiki.opendaylight.org/display/ODL/2020-05-27+Integration+Meeting+Minutes just to remind you, the names of netvirt jobs will always have the ODL version Thanks,
|
|
|
|
integration meeting minutes (6/3/2020)
JamO Luhrsen
|
|
|
|
Re: integration meeting minutes (5/27/2020)
JamO Luhrsen
On 6/3/20 7:21 PM, Anil Belur wrote:
Anil, the list is there in the meeting minutes: https://wiki.opendaylight.org/display/ODL/2020-05-27+Integration+Meeting+Minutes just to remind you, the names of netvirt jobs will always have
the ODL version Thanks,
|
|
|
|
Re: integration meeting minutes (5/27/2020)
Anil Belur
Hi Srinivas, As discussed on the integration call, please share the excel sheet which describes the list of job names that need to be modified as per the table shared on the meeting minutes. Thanks, Anil On Thu, May 28, 2020 at 2:52 PM JamO Luhrsen <jluhrsen@...> wrote: https://wiki.opendaylight.org/display/ODL/2020-05-27+Integration+Meeting+Minutes |
|
|
|
Re: Gerrit verification is failing due to python dependency issues.
Srinivas <srinivas.rachakonda@...>
Thanks a lot Thanh.
Thanks, Srinivas +91-9243478719
From: Thanh Ha <zxiiro@...>
Sent: 02 June 2020 03:04 To: Srinivas Rachakonda <srinivas.rachakonda@...> Cc: Anil Belur <abelur@...>; JamO Luhrsen <jluhrsen@...>; odl netvirt dev <netvirt-dev@...>; P S Ravikanth <ravikanth.ps@...>; karthikeyan.k@...; integration-dev@... Subject: Re: [integration-dev] Gerrit verification is failing due to python dependency issues.
Hi Srinivas,
Your patch was 45 commits away from master. I rebased the patch to make it catch up with master and the verify job is passing now.
It is good practice to always do a "git fetch" and "git rebase origin/master" before pushing commits Gerrit to ensure your work is based on the latest and greatest. Often simply rebasing your patch to ensure it's on the latest and greatest will help solve some build issues too.
Hope this helps, Thanh
On Mon, Jun 1, 2020 at 1:38 PM JamO Luhrsen <jluhrsen@...> wrote:
|
|
|
|
Re: Gerrit verification is failing due to python dependency issues.
Thanh ha <zxiiro@...>
Hi Srinivas, Your patch was 45 commits away from master. I rebased the patch to make it catch up with master and the verify job is passing now. It is good practice to always do a "git fetch" and "git rebase origin/master" before pushing commits Gerrit to ensure your work is based on the latest and greatest. Often simply rebasing your patch to ensure it's on the latest and greatest will help solve some build issues too. Hope this helps, Thanh On Mon, Jun 1, 2020 at 1:38 PM JamO Luhrsen <jluhrsen@...> wrote:
|
|
|
