integration-dev@lists.opendaylight.org | Upgrade script - CSIT

Home Messages Hashtags Subgroups

Date Date 1 - 2 of 2

Upgrade script - CSIT

Srinivas <srinivas.rachakonda@...> #14170 ++ Netvirt-Dev Thanks, Srinivas +91-9243478719 toggle quoted message Show quoted text From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 05 June 2020 11:13 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'karthikeyan.k@...' <karthikeyan.k@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT Hi Dheena, Thanks for the debug session we had today. The issue here is the bundle came up around 1min 58sec, while my script looks for 1min. Now the question still remains if we need to wait more time for bundle to come up in Aluminium, while for Magnesium it came up within 1 min. So today’s action items are: Srinivas - will increase the timeout and test it. Dheena – Will check the difference between Magnesium and Aluminium to find the timing issue. Below is bundle URL. Magnesium: https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upgrade-snat-conntrack-magnesium/263 Magnesium ODL: https://nexus.opendaylight.org/content/repositories//autorelease-4112/org/opendaylight/integration/karaf/0.12.2/karaf-0.12.2.zip Thanks, Srinivas +91-9243478719 From: Dheenadayalan B <dheenadayalan.b@...> Sent: 04 June 2020 20:58 To: srinivas.rachakonda@... Cc: 'P S Ravikanth' <ravikanth.ps@...>; karthikeyan.k@...; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT Hi Srini, RPC will be registered on openflowdevice(DPN) connected with the controller. From the logs provided observerved rpc call executed even before dpn 259791131554203 connection established, hence the error. Rpc call execution time: 20200604 07:58:56.147 / 20200604 07:58:56.160 / 00:00:00.013 (2020-06-04T02:28:56) (UTC+05:30hrs) body={ "input": { "node": "/opendaylight-inventory:nodes/opendaylight-inventory:node[opendaylight-inventory:id='openflow:259791131554203']", "node-id": 259791131554203 } CSIT logs: (UTC time zone) 2020-06-04T02:29:56,797 \| INFO \| epollEventLoopGroup-7-3 \| ContextChainHolderImpl \| 382 - org.opendaylight.openflowplugin.impl - 0.11.0.SNAPSHOT \| Device openflow:259791131554203 connected. Could you please delay script execution by one minute and check if the same issue occurring again? Thanks, Dheena From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 04 June 2020 09:29 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; karthikeyan.k@...; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT https://jira.opendaylight.org/browse/NETVIRT-1672 Thanks, Srinivas +91-9243478719 From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 04 June 2020 09:19 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'karthikeyan.k@...' <karthikeyan.k@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT Please see the below error: POST Response : url=http://10.30.170.113:8181/restconf/operations/arbitrator-reconcile:get-active-bundle status=501, reason=Not Implemented Can this be added to Aluminium. We have aluminium release in next 15 days and this fix is needed. Thanks, Srinivas +91-9243478719 From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 04 June 2020 09:16 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'karthikeyan.k@...' <karthikeyan.k@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT Hi Dheena, The fixes passed in Magnesium but failed in Aluminium. Please kindly look into this on priority and let me know if the below json needs to be changed for Aluminium. https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upgrade-snat-conntrack-aluminium/88/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upgrade-snat-conntrack-aluminium/88/robot-plugin/log_full.html.gz Thanks, Srinivas +91-9243478719 From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 01 June 2020 14:01 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'karthikeyan.k@...' <karthikeyan.k@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT Hi Dheena, Thanks a lot for the help. With your suggestions I was able to run the scripts and it went fine. Can you please mail me in details the reasons for the changes so that I can ask the community for acceptance. Thanks, Srinivas +91-9243478719 From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 01 June 2020 10:49 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'karthikeyan.k@...' <karthikeyan.k@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: Upgrade script - CSIT Hi Dheena, I am working on reproducing the upgrade issue in CSIT today. I will ping once I am at failed state. We will debug it together on my local setup. Thanks, Srinivas +91-9243478719 smime.p7s smime.p7s
More All Messages By This Member
Dheenadayalan B <dheenadayalan.b@...> #14172 Hi Srini/Karthik/Devteam, Unlike in magnesium branch, northbound interface addition leads to time delay of more than one minute, leads to upgrade csit failure, Openflowplugin module was listening to port 6653/6633 , however connection establishment between ovs and plugin made with a delay of one minute through northbound. logs as below: Openflowplugin startup completed as below: Karaf.log: 020-06-04T02:28:15,339 \| INFO \| SystemReadyService-0 \| OpenFlowPluginProviderImpl \| 382 - org.opendaylight.openflowplugin.impl - 0.11.0.SNAPSHOT \| onSystemBootReady() received, starting the switch connections 2020-06-04T02:28:15,547 \| INFO \| OFP-SwitchConnectionProvider-Udp/TcpHandler_6653-0 \| TcpHandler \| 392 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.11.0.SNAPSHOT \| Switch listener started and ready to accept incoming tcp/tls connections on port: 6653 2020-06-04T02:28:15,550 \| INFO \| OFP-SwitchConnectionProvider-Udp/TcpHandler_6653-0 \| OpenflowDiagStatusProviderImpl \| 392 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.11.0.SNAPSHOT \| The report status is OPERATIONAL for OPENFLOW_SERVER_6653 2020-06-04T02:28:15,559 \| INFO \| SystemReadyService-0 \| SouthboundProvider \| 401 - org.opendaylight.ovsdb.southbound-impl - 1.11.0.SNAPSHOT \| Started OVSDB Manager (in system ready listener) 2020-06-04T02:28:15,561 \| INFO \| OFP-SwitchConnectionProvider-Udp/TcpHandler_6633-0 \| OpenFlowPluginProviderImpl \| 382 - org.opendaylight.openflowplugin.impl - 0.11.0.SNAPSHOT \| All switchConnectionProviders are up and running (2). 2020-06-04T02:28:15,561 \| INFO \| OFP-SwitchConnectionProvider-Udp/TcpHandler_6633-0 \| OpenflowDiagStatusProviderImpl \| 392 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.11.0.SNAPSHOT \| The report status is OPERATIONAL for OPENFLOW Script for northbound setup executed @20200604 07:58:43.060 (UTC+05:30) (2020-06-04T02:28:43) sudo ovs-vsctl set-manager tcp:10.30.170.113:6640 ptcp:6641:127.0.0.1 karaf.log: 2020-06-04T02:28:43,942 \| DEBUG \| jobcoordinator-main-task-6 \| InterfaceTopologyStateListener \| 248 - org.opendaylight.genius.interfacemanager-impl - 0.9.0.SNAPSHOT \| adding bridge references for bridge: OvsdbBridgeAugmentation{getBridgeExternalIds={BridgeExternalIdsKey{_bridgeExternalIdKey=created_by}=BridgeExternalIds{getBridgeExternalIdKey=created_by, getBridgeExternalIdValue=odl, augmentation=[]}, BridgeExternalIdsKey{_bridgeExternalIdKey=opendaylight-iid}=BridgeExternalIds{getBridgeExternalIdKey=opendaylight-iid, getBridgeExternalIdValue=/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/758da02d-7c35-43b6-99aa-c9c57108210f/bridge/br-int'], augmentation=[]}}, getBridgeName=OvsdbBridgeName{_value=br-int}, getBridgeOtherConfigs={BridgeOtherConfigsKey{_bridgeOtherConfigKey=hwaddr}=BridgeOtherConfigs{getBridgeOtherConfigKey=hwaddr, getBridgeOtherConfigValue=ec:47:56:1c:b1:9b, augmentation=[]}, BridgeOtherConfigsKey{_bridgeOtherConfigKey=disable-in-band}=BridgeOtherConfigs{getBridgeOtherConfigKey=disable-in-band, getBridgeOtherConfigValue=true, augmentation=[]}}, getBridgeUuid=Uuid{_value=935221ad-40ad-4aaa-939c-a8654e883bd8}, getDatapathId=DatapathId{_value=00:00:ec:47:56:1c:b1:9b}, getDatapathType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.DatapathTypeSystem, getFailMode=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbFailModeSecure, getManagedBy=OvsdbNodeRef{_value=KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology, org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey{_topologyId=Uri{_value=ovsdb:1}}], org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey{_nodeId=Uri{_value=ovsdb://uuid/758da02d-7c35-43b6-99aa-c9c57108210f}}]]}}, getProtocolEntry={ProtocolEntryKey{_protocol=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbBridgeProtocolOpenflow13}=ProtocolEntry{getProtocol=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbBridgeProtocolOpenflow13, augmentation=[]}}, isStpEnable=false}, dpn: 259791131554203 Openflowplugin and ovs communication time: Karaf.log: 2020-06-04T02:29:56,797 \| INFO \| epollEventLoopGroup-7-3 \| ContextChainHolderImpl \| 382 - org.opendaylight.openflowplugin.impl - 0.11.0.SNAPSHOT \| Device openflow:259791131554203 connected. Ovs-vswitchd.log: 2020-06-04T02:29:56.768Z\|01001\|rconn\|INFO\|br-int<->tcp:10.30.170.113:6653: connected Thanks, Dheena toggle quoted message Show quoted text From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 05 June 2020 11:31 To: 'Dheenadayalan B' <dheenadayalan.b@...>; 'odl netvirt dev' <netvirt-dev@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; karthikeyan.k@...; 'Chetan Arakere Gowdru' <chetan.arakere@...>; integration-dev@... Subject: RE: Upgrade script - CSIT ++ Netvirt-Dev Thanks, Srinivas +91-9243478719 From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 05 June 2020 11:13 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'karthikeyan.k@...' <karthikeyan.k@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT Hi Dheena, Thanks for the debug session we had today. The issue here is the bundle came up around 1min 58sec, while my script looks for 1min. Now the question still remains if we need to wait more time for bundle to come up in Aluminium, while for Magnesium it came up within 1 min. So today’s action items are: Srinivas - will increase the timeout and test it. Dheena – Will check the difference between Magnesium and Aluminium to find the timing issue. Below is bundle URL. Magnesium: https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upgrade-snat-conntrack-magnesium/263 Magnesium ODL: https://nexus.opendaylight.org/content/repositories//autorelease-4112/org/opendaylight/integration/karaf/0.12.2/karaf-0.12.2.zip Thanks, Srinivas +91-9243478719 From: Dheenadayalan B <dheenadayalan.b@...> Sent: 04 June 2020 20:58 To: srinivas.rachakonda@... Cc: 'P S Ravikanth' <ravikanth.ps@...>; karthikeyan.k@...; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT Hi Srini, RPC will be registered on openflowdevice(DPN) connected with the controller. From the logs provided observerved rpc call executed even before dpn 259791131554203 connection established, hence the error. Rpc call execution time: 20200604 07:58:56.147 / 20200604 07:58:56.160 / 00:00:00.013 (2020-06-04T02:28:56) (UTC+05:30hrs) body={ "input": { "node": "/opendaylight-inventory:nodes/opendaylight-inventory:node[opendaylight-inventory:id='openflow:259791131554203']", "node-id": 259791131554203 } CSIT logs: (UTC time zone) 2020-06-04T02:29:56,797 \| INFO \| epollEventLoopGroup-7-3 \| ContextChainHolderImpl \| 382 - org.opendaylight.openflowplugin.impl - 0.11.0.SNAPSHOT \| Device openflow:259791131554203 connected. Could you please delay script execution by one minute and check if the same issue occurring again? Thanks, Dheena From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 04 June 2020 09:29 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; karthikeyan.k@...; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT https://jira.opendaylight.org/browse/NETVIRT-1672 Thanks, Srinivas +91-9243478719 From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 04 June 2020 09:19 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'karthikeyan.k@...' <karthikeyan.k@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT Please see the below error: POST Response : url=http://10.30.170.113:8181/restconf/operations/arbitrator-reconcile:get-active-bundle status=501, reason=Not Implemented Can this be added to Aluminium. We have aluminium release in next 15 days and this fix is needed. Thanks, Srinivas +91-9243478719 From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 04 June 2020 09:16 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'karthikeyan.k@...' <karthikeyan.k@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT Hi Dheena, The fixes passed in Magnesium but failed in Aluminium. Please kindly look into this on priority and let me know if the below json needs to be changed for Aluminium. https://jenkins.opendaylight.org/releng/view/netvirt-csit/job/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upgrade-snat-conntrack-aluminium/88/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upgrade-snat-conntrack-aluminium/88/robot-plugin/log_full.html.gz Thanks, Srinivas +91-9243478719 From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 01 June 2020 14:01 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'karthikeyan.k@...' <karthikeyan.k@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: RE: Upgrade script - CSIT Hi Dheena, Thanks a lot for the help. With your suggestions I was able to run the scripts and it went fine. Can you please mail me in details the reasons for the changes so that I can ask the community for acceptance. Thanks, Srinivas +91-9243478719 From: srinivas.rachakonda@... <srinivas.rachakonda@...> Sent: 01 June 2020 10:49 To: 'Dheenadayalan B' <dheenadayalan.b@...> Cc: 'P S Ravikanth' <ravikanth.ps@...>; 'karthikeyan.k@...' <karthikeyan.k@...>; 'Chetan Arakere Gowdru' <chetan.arakere@...> Subject: Upgrade script - CSIT Hi Dheena, I am working on reproducing the upgrade issue in CSIT today. I will ping once I am at failed state. We will debug it together on my local setup. Thanks, Srinivas +91-9243478719 smime.p7s smime.p7s
More

1 - 2 of 2

1

Previous Topic Next Topic

Home Hashtags Subgroups Terms