Re: Netvirt patch tests failing on Sodium


Jamo Luhrsen <jluhrsen@...>
 

This is exactly the idea of the apex snapshots. It works. It saves time and gives
us a tested and stable VM to just bring up and then test whatever ODL version
we want. We have automatic gate jobs running, like this [0], which are stable.

problem is nobody is working on it any more. It needs people for maintenance,
improvements, monitoring.

As for doing this same thing with devstack, I've been told in the past that
devstack VMs don't work as expected after a reboot. So saving a snapshot and
bringing it up would be the same thing. It's something we've discussed long
back and dismissed. I may be remembering wrong.

JamO

[0] https://jenkins.opendaylight.org/releng/job/netvirt-csit-1node-0cmb-1ctl-2cmp-apex-queens-gate-snat-conntrack-neon/

On 3/22/19 6:02 AM, Dayavanti Gopal Kamath wrote:
We really need to change our csits to work with a stable working snapshot of the devstack vm with all stacking done. Can we not just stack, store the vm image somewhere, if no other storage is available, neutron git repository, and pull it from there for every run? It will help cut down test execution time by 30 minutes and shield us from this unnecessary pain. We can choose when to update the vm image, after testing.
Thanks,
daya
*From:* netvirt-dev-bounces@... [mailto:netvirt-dev-bounces@...] *On Behalf Of *Jaya Priyadarshini
*Sent:* Friday, March 22, 2019 3:18 PM
*To:* Faseela K <faseela.k@...>; sam Hague <shague@...>; Jamo Luhrsen <jluhrsen@...>; Venkatrangan Govindarajan <vgovindarajan@...>
*Cc:* netvirt-dev@...; integration-dev@...
*Subject:* Re: [netvirt-dev] Netvirt patch tests failing on Sodium
Hi Faseela,
Few more observations,
(i)Same jobs passes for sodium but fails for neon and fluorine .
(ii)Some jobs have different failure reason as well, apart from rabbitmq issue. They are as below:
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-neon/1158/compute_1/stack.log.gz
Stack stops abruptly at Downloadinghttps://files.pythonhosted.org/packages/e7/1e/3dcca007f974fe4eb369bf1b8629d5e342bb3055e2001b2e5340aaefae7a/uwsgi-2.0.18.tar.gz (
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-1cmb-0ctl-0cmp-openstack-queens-tempest-stateful-snat-conntrack-neon/187/console.log.gz
stack fails and there is no stack.log also (no compute folder itself).
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-sfc-neon/187/console.log.gz
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-neon/221/console.log.gz
Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 10.30.168.3, 2604:e100:1:0:f816:3eff:fe45:48d6
Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|10.30.168.3|:443... connected.
HTTP request sent, awaiting response... Extracting the new controller...
404 Not Found
Can anyone please help?...
Regards
Jaya
*From:*Jaya Priyadarshini
*Sent:* Friday, March 22, 2019 2:40 PM
*To:* Faseela K; integration-dev@... <mailto:integration-dev@...>
*Cc:* netvirt-dev@... <mailto:netvirt-dev@...>
*Subject:* RE: Netvirt patch tests failing on Sodium
Hi Faseela,
All the jobs in netvirt and sandbox are failing due to this
+ ssh -t -t 10.30.170.79 'sudo rabbitmqctl list_vhosts'
*10:34:52*Warning: Permanently added '10.30.170.79' (ECDSA) to the list of known hosts.
*10:34:52*Connection to 10.30.170.79 closed.
*10:34:52*+ grep nova_cell1 rabbit.txt
*10:34:52*+ rc=1
*10:34:52*+ (( 1 == 0 ))
*10:34:52*+ (( 60 == 60 ))
*10:34:52*+ break
*10:34:52*+ return 1
It is trying for 60 attempts and fails, not sure which change could have done this.
Stack log says this change has gone in.
DevStack Version: queens
Change: e98e7d5ae3df3ba1ccb2b1a2af649a853d05e56b Merge "Enable nova VNC configuration on the subnode" into stable/queens 2019-02-27 14:00:34 +0000
Can anyone please help resolving this.
Regards
Jaya
*From:*Faseela K
*Sent:* Thursday, March 21, 2019 4:51 PM
*To:* integration-dev@... <mailto:integration-dev@...>; Jaya Priyadarshini
*Cc:* netvirt-dev@... <mailto:netvirt-dev@...>
*Subject:* Netvirt patch tests failing on Sodium
Jaya/Venkat,
   Anybody has an idea why the patch tests are failing?
https://git.opendaylight.org/gerrit/#/c/75427/
  See below errors in console :
*16:05:03*tar: Removing leading `/' from member names
*16:05:03*tar: /tmp/zrpcd.init.log: Cannot stat: No such file or directory
*16:05:03*tar: Exiting with failure status due to previous errors
*16:05:03*Connection to 10.30.170.37 closed.
Thanks,
Faseela
https://jenkins.opendaylight.org/releng/job/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-sodium/11/console

Join integration-dev@lists.opendaylight.org to automatically receive all group messages.