Thanks Jaya, see inline...

Hi Jamo,

 

There are 5 blockers for netvirt in Sodium-SR1 which are of 2 category:

1. 2 jobs of dcgw –Both the jobs has 3 failures

1 fails for exceptions caught in teardown : broken recently

https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/builder-copy-sandbox-logs/685/JP-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/4/robot-plugin/log_full.html.gz

no exception in sept12 result.

Can someone figure out the severity of this newly introduced exception? does it affect
end-user functionality?

 

2 fails saying association of vpn fails  but surprisingly when I ran the same suite in sandbox it passes totally.

https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/JP-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/5/robot-plugin/log_full.html.gz

              Hence something is being left over from previous suite or something like that, needs fix.

              But not a blocker clearly.

ok, can you mark it's JIRA as non-blocking? I would suggest someone spend some
time running the previous suites + failing suite in the sandbox to try to narrow down
the root cause. Also, need to make this jira non-blocker status.

 

2. 3 jobs of rocky with openstack issue : was existing since beginning.

Well the same testcase passes for 0cmb cases, ie, compute and controller separate so clearly not a netvirt functionality issue or major openstack issue.

So this is not a regression since it was failing the same way with original Sodium release right?

I remember when I was working on netvirt jobs that we had some bugs that only showed
up when compute nodes were or were not combined with the control node. My guess is
that it's a legit netvirt or openstack bug.

Thanks,
JamO

 

Regards

Jaya

From: Jamo Luhrsen <jluhrsen@...>
Sent: Tuesday, October 29, 2019 10:35 PM
To: Jaya Priyadarshini <jaya.priyadarshini@...>; Abhinav Gupta <abhinav.gupta@...>; Daniel De La Rosa <ddelarosa@...>; Srinivas Rachakonda <srinivas.rachakonda@...>
Cc: integration-dev@...
Subject: Re: [integration-dev] Sodium SR1 release candidate

 

Hey guys,

at this point, blocker bugs are "blocking" an already late release.
We need to get them resolved asap.

1) are they true regressions and issues that did not exist with the
   original Sodium release?

2) ignoring 1) above, are the bugs truly "blocker" in nature in that
   the netvirt functionality is broken and cannot be used?

Thanks,
JamO

On 10/29/19 12:02 AM, Jaya Priyadarshini wrote:

Hi Abhinav/Srinivas,

 

BLOCKER	jamo	netvirt	https://jenkins.opendaylight.org/releng/job/netvirt-csit-1node-1cmb-0ctl-0cmp-openstack-rocky-upstream-stateful-itm-direct-tunnels-sodium/93/
BLOCKER	jamo	netvirt	https://jenkins.opendaylight.org/releng/job/netvirt-csit-1node-1cmb-0ctl-0cmp-openstack-rocky-upstream-stateful-snat-conntrack-sodium/93/
BLOCKER	jamo	netvirt	https://jenkins.opendaylight.org/releng/job/netvirt-csit-1node-1cmb-0ctl-0cmp-openstack-rocky-upstream-stateful-sodium/93/

 

The above 3 jobs when run witj 0cmb-1ctl-2cmp cases have 100% pass result and have 2 failures with 1cmb-0ctl-0cmb cases.

Not test code issue but odl code issue.

Can anyone have a look?

 

Regards

Jaya

From: integration-dev@... <integration-dev@...> On Behalf Of JamO Luhrsen via Lists.Opendaylight.Org
Sent: Thursday, October 24, 2019 2:24 AM
To: Abhinav Gupta <abhinav.gupta@...>; Daniel De La Rosa <ddelarosa@...>; Srinivas Rachakonda <srinivas.rachakonda@...>
Cc: integration-dev@...
Subject: Re: [integration-dev] Sodium SR1 release candidate

 

We need to get netvirt under control at this point. They have a lot of jobs,
and were never fully healthy but without maintenance they are going to create
a lot of work every release cycle.

I spent some time today going through our sodium sr1 failures and made progress
on the sheet:

https://docs.google.com/spreadsheets/d/1CN-ACdIekRKMYRIuREIRALl6S8az7quAcQlRb3KeMJ8/edit?pli=1#gid=1864238939

there are three jobs with a basic netvirt failure that was ignored in
the original sodium release (openstack instance not getting an ip address)
which seems like a fundamental failure. The original sodium release let
it through under the assumption that it was a test code problem. I don't
think it is. I marked those as BLOCKER.

Should we mark the DCGW jobs as ignore? I don't have time to debug them
and they are saw-tooth failures and inconsistent. No idea if they are
regressions or not.

beyond netvirt, I'm following up on one openflowplugin job and once those
are understood we will only be left with netvirt to be done.

Thanks,
JamO

On 10/21/19 11:41 PM, Abhinav Gupta wrote:

Srinivas,

Please look into the CSIT jobs for failures. Please take help from Karthika if required. I’m in parallel onboarding myself with the overall Netvirt CSIT. Let’s discuss and get this moving.
I see 20 CSIT jobs for NetVirt. We may want to eliminate the unnecessary jobs as well.

 

Regards,
Abhinav

 

From: Daniel De La Rosa <ddelarosa@...>
Sent: Tuesday, October 22, 2019 1:49 AM
To: JamO Luhrsen <jluhrsen@...>; Abhinav Gupta <abhinav.gupta@...>
Cc: Release <release@...>; tsc@...; integration-dev@...; netvirt-dev@...; Bgpcep-Dev <bgpcep-dev@...>; controller-dev@...; lispflowmapping-dev@...; netconf-dev@...; openflowplugin-dev@...
Subject: Re: [integration-dev] Sodium SR1 release candidate

 

Team, 

 

Friendly reminder to review Sodium SR1 tracking sheet so we can release it on time.

 

 

https://docs.google.com/spreadsheets/d/1CN-ACdIekRKMYRIuREIRALl6S8az7quAcQlRb3KeMJ8/edit?pli=1#gid=1864238939

 

 

 

Thanks

 

On Fri, Oct 18, 2019 at 11:28 AM JamO Luhrsen <jluhrsen@...> wrote:

In our weekly integration call we decided to use the most recent successful
Sodium autorelease build as our release candidate for SR1 which is due next
week on October 24th.

Here is the build:
https://nexus.opendaylight.org/content/repositories//autorelease-3438/org/opendaylight/integration/karaf/0.11.1/karaf-0.11.1.zip

Here is the integration-test job using that build:
https://jenkins.opendaylight.org/releng/view/autorelease/job/integration-distribution-test-sodium/366/

Here is the tracking sheet we need to vet:
https://docs.google.com/spreadsheets/d/1CN-ACdIekRKMYRIuREIRALl6S8az7quAcQlRb3KeMJ8/edit?pli=1#gid=1864238939

Projects that have pending failures:
bgpcep, controller, lispflowmapping, netconf, netvirt, openflowplugin, ovsdb, yangtools

Thanks,
JamO

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13788): https://lists.opendaylight.org/g/integration-dev/message/13788
Mute This Topic: https://lists.opendaylight.org/mt/35075895/1964961
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ddelarosa@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

--

Daniel de la Rosa
Customer Support Manager ( ODL release manager )
Lumina Networks Inc.
e: ddelarosa@...
m:  +1 408 7728120