Re: netvirt-csit gone all red


JamO Luhrsen
 

Faseela, Jaya,

Are we only talking about a single problem now, or multiple?

The problem I replied to yesterday was a legit packaging version issue that we
had no control over and was fixed by openstack.

The problem in the link Jaya gave below is the failure to clone a devstack
repo. How often is that happening. I ran the clone in a loop on my laptop
for over 30m (so some hundreds of times) and I never got a failure. If it's
happening a lot in our infra (still) then we will need Anil's help to
figure it out.

Or are there other random problems?

Thanks,
JamO

On 7/25/19 4:51 AM, Faseela K wrote:

Anil,

 

      Looks like the connectivity towards openstack infra from our CSIT VMs have some issues.

      When we try from our local servers, we are not hitting the issue.

      We have been blocked because of this for quite sometime now.

      How do we take this forward now?

     

      

Thanks,

Faseela

 

From: Jaya Priyadarshini <jaya.priyadarshini@...>
Sent: Thursday, July 25, 2019 3:27 PM
To: Anil Belur <abelur@...>
Cc: Faseela K <faseela.k@...>; netvirt-dev@...; 'integration-dev@...' (integration-dev@...) (integration-dev@...) <integration-dev@...>
Subject: RE: [integration-dev] netvirt-csit gone all red

 

Hi Anil,

 

I am able to do git clone locally.

 

root@ubuntu:~# git clone https://git.openstack.org/openstack-dev/devstack --branch stable/queens

Cloning into 'devstack'...

remote: Enumerating objects: 43869, done.

remote: Counting objects: 100% (43869/43869), done.

remote: Compressing objects: 100% (12754/12754), done.

remote: Total 43869 (delta 31336), reused 42513 (delta 30435)

Receiving objects: 100% (43869/43869), 8.34 MiB | 2.56 MiB/s, done.

Resolving deltas: 100% (31336/31336), done.

Checking connectivity... done.

root@ubuntu:~#

 

Regards

Jaya

 

From: Anil Belur <abelur@...>
Sent: Thursday, July 25, 2019 3:05 PM
To: Jaya Priyadarshini <jaya.priyadarshini@...>
Cc: Faseela K <faseela.k@...>; netvirt-dev@...; 'integration-dev@...' (integration-dev@...) (integration-dev@...) <integration-dev@...>
Subject: Re: [integration-dev] netvirt-csit gone all red

 

I think this might be an issue with the openstack infra itself, and not on our infra. I'm seeing the git clone issue locally.

 

Cloning into 'devstack'...
fatal: unable to access 'https://git.openstack.org/openstack-dev/devstack/': Failed to connect to opendev.org port 443: No route to host

 

- Anil 

 

On Thu, Jul 25, 2019 at 5:51 PM Anil Belur <abelur@...> wrote:

Greetings Jaya:

 

Can you raise an issue at https://support.linuxfoundation.org about it please?

 

Thanks,

Anil

 

On Thu, Jul 25, 2019 at 5:29 PM Jaya Priyadarshini <jaya.priyadarshini@...> wrote:

Anil,

 

Can u check this, this is seen in this job.

https://jenkins.opendaylight.org/releng/job/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-sodium/109/

https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-sodium/111/control_1/get_devstack.sh.txt.gz

 

All jobs of this link has this issue.

 

fatal: unable to access 'https://git.openstack.org/openstack-dev/devstack/': Empty reply from server

/tmp/get_devstack.sh: line 19: cd: devstack: No such file or directory

 

But I am able to access the same from all my local machines, only cloud computes are having this issue.

 

Regards

Jaya

From: integration-dev <integration-dev-bounces@...> On Behalf Of Jaya Priyadarshini
Sent: Tuesday, July 23, 2019 4:16 PM
To: Faseela K <faseela.k@...>; netvirt-dev@...; 'integration-dev@...' (integration-dev@...) (integration-dev@...) <integration-dev@...>
Subject: Re: [integration-dev] netvirt-csit gone all red

 

Well, found the reason behind this:

 

git clone https://git.openstack.org/openstack-dev/devstack --branch stable/rocky

Cloning into 'devstack'...

fatal: unable to access 'https://git.openstack.org/openstack-dev/devstack/': Encountered end of file

/tmp/get_devstack.sh: line 19: cd: devstack: No such file or directory

--2019-07-23 00:58:49--  https://raw.githubusercontent.com/shague/odl_tools/master/fix-logging.patch.txt

Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.136.133

Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.136.133|:443... connected.

HTTP request sent, awaiting response... 200 OK

Length: 2156 (2.1K) [text/plain]

 

--2019-07-23 00:58:49--  https://raw.githubusercontent.com/shague/odl_tools/master/fix-logging.patch.txt by Sam is not working anymore.

And this problem is existing for all devstack branches , be it queens or rocky.

Will be fixing it.

 

Regards

Jaya

 

 

From: Faseela K <faseela.k@...>
Sent: Tuesday, July 23, 2019 3:08 PM
To: Jaya Priyadarshini <jaya.priyadarshini@...>; netvirt-dev@...; 'integration-dev@...' (integration-dev@...) (integration-dev@...) <integration-dev@...>
Subject: RE: netvirt-csit gone all red

 

Hi everyone,

   Please stop triggering patch tests for netvirt until we fix the issue.

Thanks,

Faseela

 

From: netvirt-dev <netvirt-dev-bounces@...> On Behalf Of Jaya Priyadarshini
Sent: Tuesday, July 23, 2019 12:55 PM
To: netvirt-dev@...; 'integration-dev@...' (integration-dev@...) (integration-dev@...) <integration-dev@...>
Subject: [netvirt-dev] netvirt-csit gone all red

 

Hi All,

 

Almost all last build have failed with below error in netvirt

 

+ ssh 10.30.170.83 'sed -i '\''s/flat_networks public/flat_networks public,physnet1/'\'' /opt/stack/devstack/lib/neutron'

Warning: Permanently added '10.30.170.83' (ECDSA) to the list of known hosts.

sed: can't read /opt/stack/devstack/lib/neutron: No such file or directory

++ trap_handler 833 15569

++ local prog=/tmp/jenkins3447639189848961768.sh

++ local lastline=833

++ local lasterr=15569

++ echo 'trap_handler: /tmp/jenkins3447639189848961768.sh: line 833: exit status of last command: 15569'

trap_handler: /tmp/jenkins3447639189848961768.sh: line 833: exit status of last command: 15569

++ echo 'trap_handler: command: ssh ${!CONTROLIP} "sed -i '\''s/flat_networks public/flat_networks public,physnet1/'\'' /opt/stack/devstack/lib/neutron"'

trap_handler: command: ssh ${!CONTROLIP} "sed -i 's/flat_networks public/flat_networks public,physnet1/' /opt/stack/devstack/lib/neutron"

++ exit 1

Build step 'Execute shell' marked build as failure

$ ssh-agent -k

 

Trying to see if I can fix it, Anyone has any idea what could have gone wrong?

 

Regards

Jaya

_______________________________________________
integration-dev mailing list
integration-dev@...
https://lists.opendaylight.org/mailman/listinfo/integration-dev


_______________________________________________
integration-dev mailing list
integration-dev@...
https://lists.opendaylight.org/mailman/listinfo/integration-dev

Join integration-dev@lists.opendaylight.org to automatically receive all group messages.