[release] [netvirt-dev] COE and Netvirt CSIT red


Abhijit Kumbhare
 

Thanks for your efforts on this Jaya.


On Tue, Aug 20, 2019 at 10:59 AM Jaya Priyadarshini <jaya.priyadarshini@...> wrote:
Hi Faseela/Jamo,

I have tried multiple fixes but I m still stuck since many patches have gone in.
, I think I will one more day to fix it
but in the meantime can we just run the distribution build mentioned in the NEON SR release excel sheet with the older builder repo.
Our goal is to check distro rather than repo.
Let me know, then I will run all the jobs mentioned and will verify the distribution in sandbox.

Regards
Jaya
-----Original Message-----
From: Jaya Priyadarshini
Sent: Tuesday, August 20, 2019 12:09 PM
To: Faseela K <faseela.k@...>; Jamo Luhrsen <jluhrsen@...>; netvirt-dev@...; integration-dev@...; Release <release@...>
Subject: RE: [netvirt-dev] COE and Netvirt CSIT red

Hi Faseela,

What I suspected was right, I reverted many patches and it works https://jenkins.opendaylight.org/sandbox/job/JP-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/console
Hence it was not stack issue, stack was not even completing and job hitting exit.

Need to figure out exactly which patch was causing failure, will update asap.

Regards
Jaya
-----Original Message-----
From: Jaya Priyadarshini
Sent: Tuesday, August 20, 2019 11:20 AM
To: Faseela K <faseela.k@...>; Jamo Luhrsen <jluhrsen@...>; netvirt-dev@...; integration-dev@...; Release <release@...>
Subject: RE: [netvirt-dev] COE and Netvirt CSIT red

Jamo, Faseela,

Netvirt csit is failing since Aug8 and I see that on Aug8 , a lot of patches have gone in builder.
I have doubt on this patch,
https://git.opendaylight.org/gerrit/c/releng/builder/+/83508
Resolve ShellCheck for deploy-controller-run-test

Trying some workarounds, will update you asap.
Its failing in stack but stack is not completed even and it says stack failed and exits.
Some checks to wait for stack is missing is what I am suspecting.

Also Aug 8 , branch cut had happened so many patches have gone in.
And also not every jobs of rocky and queens are failing (including sodium, neon and fluorine) but a lot of them are So it is tricky to find the solution.

Regards
Jaya


-----Original Message-----
From: integration-dev <integration-dev-bounces@...> On Behalf Of Faseela K
Sent: Tuesday, August 20, 2019 10:48 AM
To: Jamo Luhrsen <jluhrsen@...>; netvirt-dev@...; integration-dev@...; Release <release@...>
Subject: Re: [integration-dev] [netvirt-dev] COE and Netvirt CSIT red

Jamo,
   For netvirt, Jaya is investigating the issue, and she is suspecting a releng/builder patch for the issue of stack failure.
   She will soon update.
Thanks,
Faseela

-----Original Message-----
From: netvirt-dev <netvirt-dev-bounces@...> On Behalf Of Jamo Luhrsen
Sent: Saturday, August 17, 2019 4:11 AM
To: netvirt-dev@...; integration-dev@...; Release <release@...>
Subject: [netvirt-dev] COE and Netvirt CSIT red

Hi Netvirt,

I was scanning through the csit results we need to vet for the Neon
SR2 release [0] and see that COE csit is 100% failing for a while now. And looks like netvirt jobs are failng to stack.

we don't want to release with a regression. Can you see what's really wrong and update the sheet?

Thanks,
JamO

[0]
https://protect2.fireeye.com/url?k=6781bf13-3b08109e-6781ff88-0cc47ad93de2-604a936d3d641e03&q=1&u=https%3A%2F%2Fdocs.google.com%2Fspreadsheets%2Fd%2F1sIoApv2fFp0wJcPK7m6lQDtjbUonRANrs5upSeD_DyI%2Fedit%3Fpli%3D1%23gid%3D698183592
_______________________________________________
netvirt-dev mailing list
netvirt-dev@...
https://protect2.fireeye.com/url?k=7be0a208-27690d85-7be0e293-0cc47ad93de2-d638cf0f47074d58&q=1&u=https%3A%2F%2Flists.opendaylight.org%2Fmailman%2Flistinfo%2Fnetvirt-dev
_______________________________________________
integration-dev mailing list
integration-dev@...
https://protect2.fireeye.com/url?k=50417800-0c9574f1-5041389b-864685b2085c-7469fa861fd2871b&q=1&u=https%3A%2F%2Flists.opendaylight.org%2Fmailman%2Flistinfo%2Fintegration-dev
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release


JayaPr
 

Well, It is fixed now, it is a very embarrassing fix, someone in one of the patches had missed a space between two apostrophes in the command which does stacking.

I introduced it back and it works 😐

Will be pushing the patch for this now.

 

Regards

Jaya

From: Abhijit Kumbhare <abhijitkoss@...>
Sent: Wednesday, August 21, 2019 12:27 AM
To: Jaya Priyadarshini <jaya.priyadarshini@...>
Cc: Faseela K <faseela.k@...>; Jamo Luhrsen <jluhrsen@...>; netvirt-dev@...; integration-dev@...; Release <release@...>; Dayavanti Gopal Kamath <dayavanti.gopal.kamath@...>
Subject: Re: [release] [netvirt-dev] COE and Netvirt CSIT red

 

Thanks for your efforts on this Jaya.

 

On Tue, Aug 20, 2019 at 10:59 AM Jaya Priyadarshini <jaya.priyadarshini@...> wrote:

Hi Faseela/Jamo,

I have tried multiple fixes but I m still stuck since many patches have gone in.
, I think I will one more day to fix it
but in the meantime can we just run the distribution build mentioned in the NEON SR release excel sheet with the older builder repo.
Our goal is to check distro rather than repo.
Let me know, then I will run all the jobs mentioned and will verify the distribution in sandbox.

Regards
Jaya
-----Original Message-----
From: Jaya Priyadarshini
Sent: Tuesday, August 20, 2019 12:09 PM
To: Faseela K <faseela.k@...>; Jamo Luhrsen <jluhrsen@...>; netvirt-dev@...; integration-dev@...; Release <release@...>
Subject: RE: [netvirt-dev] COE and Netvirt CSIT red

Hi Faseela,

What I suspected was right, I reverted many patches and it works https://jenkins.opendaylight.org/sandbox/job/JP-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/console
Hence it was not stack issue, stack was not even completing and job hitting exit.

Need to figure out exactly which patch was causing failure, will update asap.

Regards
Jaya
-----Original Message-----
From: Jaya Priyadarshini
Sent: Tuesday, August 20, 2019 11:20 AM
To: Faseela K <faseela.k@...>; Jamo Luhrsen <jluhrsen@...>; netvirt-dev@...; integration-dev@...; Release <release@...>
Subject: RE: [netvirt-dev] COE and Netvirt CSIT red

Jamo, Faseela,

Netvirt csit is failing since Aug8 and I see that on Aug8 , a lot of patches have gone in builder.
I have doubt on this patch,
https://git.opendaylight.org/gerrit/c/releng/builder/+/83508
Resolve ShellCheck for deploy-controller-run-test

Trying some workarounds, will update you asap.
Its failing in stack but stack is not completed even and it says stack failed and exits.
Some checks to wait for stack is missing is what I am suspecting.

Also Aug 8 , branch cut had happened so many patches have gone in.
And also not every jobs of rocky and queens are failing (including sodium, neon and fluorine) but a lot of them are So it is tricky to find the solution.

Regards
Jaya


-----Original Message-----
From: integration-dev <integration-dev-bounces@...> On Behalf Of Faseela K
Sent: Tuesday, August 20, 2019 10:48 AM
To: Jamo Luhrsen <jluhrsen@...>; netvirt-dev@...; integration-dev@...; Release <release@...>
Subject: Re: [integration-dev] [netvirt-dev] COE and Netvirt CSIT red

Jamo,
   For netvirt, Jaya is investigating the issue, and she is suspecting a releng/builder patch for the issue of stack failure.
   She will soon update.
Thanks,
Faseela

-----Original Message-----
From: netvirt-dev <netvirt-dev-bounces@...> On Behalf Of Jamo Luhrsen
Sent: Saturday, August 17, 2019 4:11 AM
To: netvirt-dev@...; integration-dev@...; Release <release@...>
Subject: [netvirt-dev] COE and Netvirt CSIT red

Hi Netvirt,

I was scanning through the csit results we need to vet for the Neon
SR2 release [0] and see that COE csit is 100% failing for a while now. And looks like netvirt jobs are failng to stack.

we don't want to release with a regression. Can you see what's really wrong and update the sheet?

Thanks,
JamO

[0]
https://protect2.fireeye.com/url?k=6781bf13-3b08109e-6781ff88-0cc47ad93de2-604a936d3d641e03&q=1&u=https%3A%2F%2Fdocs.google.com%2Fspreadsheets%2Fd%2F1sIoApv2fFp0wJcPK7m6lQDtjbUonRANrs5upSeD_DyI%2Fedit%3Fpli%3D1%23gid%3D698183592
_______________________________________________
netvirt-dev mailing list
netvirt-dev@...
https://protect2.fireeye.com/url?k=7be0a208-27690d85-7be0e293-0cc47ad93de2-d638cf0f47074d58&q=1&u=https%3A%2F%2Flists.opendaylight.org%2Fmailman%2Flistinfo%2Fnetvirt-dev
_______________________________________________
integration-dev mailing list
integration-dev@...
https://protect2.fireeye.com/url?k=50417800-0c9574f1-5041389b-864685b2085c-7469fa861fd2871b&q=1&u=https%3A%2F%2Flists.opendaylight.org%2Fmailman%2Flistinfo%2Fintegration-dev
_______________________________________________
release mailing list
release@...
https://lists.opendaylight.org/mailman/listinfo/release


JamO Luhrsen
 

ouch! I'd suggest you use git blame and we could track down
the offender, but I'm afraid it might be me so better to just
fix it and move on :)

Good work Jaya. I know how annoying it can be.

JamO

On 8/20/19 12:26 PM, Jaya Priyadarshini wrote:
Well, It is fixed now, it is a very embarrassing fix, someone in one of the patches had missed a space between two apostrophes in the command which does stacking.
I introduced it back and it works 😐
Will be pushing the patch for this now.
Regards
Jaya
*From:* Abhijit Kumbhare <abhijitkoss@...>
*Sent:* Wednesday, August 21, 2019 12:27 AM
*To:* Jaya Priyadarshini <jaya.priyadarshini@...>
*Cc:* Faseela K <faseela.k@...>; Jamo Luhrsen <jluhrsen@...>; netvirt-dev@...; integration-dev@...; Release <release@...>; Dayavanti Gopal Kamath <dayavanti.gopal.kamath@...>
*Subject:* Re: [release] [netvirt-dev] COE and Netvirt CSIT red
Thanks for your efforts on this Jaya.
On Tue, Aug 20, 2019 at 10:59 AM Jaya Priyadarshini <jaya.priyadarshini@... <mailto:jaya.priyadarshini@...>> wrote:
Hi Faseela/Jamo,
I have tried multiple fixes but I m still stuck since many patches
have gone in.
, I think I will one more day to fix it
but in the meantime can we just run the distribution build mentioned
in the NEON SR release excel sheet with the older builder repo.
Our goal is to check distro rather than repo.
Let me know, then I will run all the jobs mentioned and will verify
the distribution in sandbox.
Regards
Jaya
-----Original Message-----
From: Jaya Priyadarshini
Sent: Tuesday, August 20, 2019 12:09 PM
To: Faseela K <faseela.k@...
<mailto:faseela.k@...>>; Jamo Luhrsen
<jluhrsen@... <mailto:jluhrsen@...>>;
netvirt-dev@...
<mailto:netvirt-dev@...>;
integration-dev@...
<mailto:integration-dev@...>; Release
<release@... <mailto:release@...>>
Subject: RE: [netvirt-dev] COE and Netvirt CSIT red
Hi Faseela,
What I suspected was right, I reverted many patches and it works
https://jenkins.opendaylight.org/sandbox/job/JP-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/console
<https://protect2.fireeye.com/url?k=c157b670-9d83b1d4-c157f6eb-860b26fb2a7b-0c13b510a96a7187&q=1&u=https%3A%2F%2Fjenkins.opendaylight.org%2Fsandbox%2Fjob%2FJP-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium%2F1%2Fconsole>
Hence it was not stack issue, stack was not even completing and job
hitting exit.
Need to figure out exactly which patch was causing failure, will
update asap.
Regards
Jaya
-----Original Message-----
From: Jaya Priyadarshini
Sent: Tuesday, August 20, 2019 11:20 AM
To: Faseela K <faseela.k@...
<mailto:faseela.k@...>>; Jamo Luhrsen
<jluhrsen@... <mailto:jluhrsen@...>>;
netvirt-dev@...
<mailto:netvirt-dev@...>;
integration-dev@...
<mailto:integration-dev@...>; Release
<release@... <mailto:release@...>>
Subject: RE: [netvirt-dev] COE and Netvirt CSIT red
Jamo, Faseela,
Netvirt csit is failing since Aug8 and I see that on Aug8 , a lot of
patches have gone in builder.
I have doubt on this patch,
https://git.opendaylight.org/gerrit/c/releng/builder/+/83508
<https://protect2.fireeye.com/url?k=ea8ed001-b65ad7a5-ea8e909a-860b26fb2a7b-5e171b7cd5c11390&q=1&u=https%3A%2F%2Fgit.opendaylight.org%2Fgerrit%2Fc%2Freleng%2Fbuilder%2F%2B%2F83508>
Resolve ShellCheck for deploy-controller-run-test
Trying some workarounds, will update you asap.
Its failing in stack but stack is not completed even and it says
stack failed and exits.
Some checks to wait for stack is missing is what I am suspecting.
Also Aug 8 , branch cut had happened so many patches have gone in.
And also not every jobs of rocky and queens are failing (including
sodium, neon and fluorine) but a lot of them are So it is tricky to
find the solution.
Regards
Jaya
-----Original Message-----
From: integration-dev
<integration-dev-bounces@...
<mailto:integration-dev-bounces@...>> On Behalf
Of Faseela K
Sent: Tuesday, August 20, 2019 10:48 AM
To: Jamo Luhrsen <jluhrsen@...
<mailto:jluhrsen@...>>;
netvirt-dev@...
<mailto:netvirt-dev@...>;
integration-dev@...
<mailto:integration-dev@...>; Release
<release@... <mailto:release@...>>
Subject: Re: [integration-dev] [netvirt-dev] COE and Netvirt CSIT red
Jamo,
   For netvirt, Jaya is investigating the issue, and she is
suspecting a releng/builder patch for the issue of stack failure.
   She will soon update.
Thanks,
Faseela
-----Original Message-----
From: netvirt-dev <netvirt-dev-bounces@...
<mailto:netvirt-dev-bounces@...>> On Behalf Of
Jamo Luhrsen
Sent: Saturday, August 17, 2019 4:11 AM
To: netvirt-dev@...
<mailto:netvirt-dev@...>;
integration-dev@...
<mailto:integration-dev@...>; Release
<release@... <mailto:release@...>>
Subject: [netvirt-dev] COE and Netvirt CSIT red
Hi Netvirt,
I was scanning through the csit results we need to vet for the Neon
SR2 release [0] and see that COE csit is 100% failing for a while
now. And looks like netvirt jobs are failng to stack.
we don't want to release with a regression. Can you see what's
really wrong and update the sheet?
Thanks,
JamO
[0]
https://protect2.fireeye.com/url?k=6781bf13-3b08109e-6781ff88-0cc47ad93de2-604a936d3d641e03&q=1&u=https%3A%2F%2Fdocs.google.com%2Fspreadsheets%2Fd%2F1sIoApv2fFp0wJcPK7m6lQDtjbUonRANrs5upSeD_DyI%2Fedit%3Fpli%3D1%23gid%3D698183592
_______________________________________________
netvirt-dev mailing list
netvirt-dev@...
<mailto:netvirt-dev@...>
https://protect2.fireeye.com/url?k=7be0a208-27690d85-7be0e293-0cc47ad93de2-d638cf0f47074d58&q=1&u=https%3A%2F%2Flists.opendaylight.org%2Fmailman%2Flistinfo%2Fnetvirt-dev
_______________________________________________
integration-dev mailing list
integration-dev@...
<mailto:integration-dev@...>
https://protect2.fireeye.com/url?k=50417800-0c9574f1-5041389b-864685b2085c-7469fa861fd2871b&q=1&u=https%3A%2F%2Flists.opendaylight.org%2Fmailman%2Flistinfo%2Fintegration-dev
_______________________________________________
release mailing list
release@... <mailto:release@...>
https://lists.opendaylight.org/mailman/listinfo/release
<https://protect2.fireeye.com/url?k=8e7ef716-d2aaf0b2-8e7eb78d-860b26fb2a7b-9ae48ddd0a369d52&q=1&u=https%3A%2F%2Flists.opendaylight.org%2Fmailman%2Flistinfo%2Frelease>


Robert Varga
 

On 20/08/2019 21:29, Jamo Luhrsen wrote:
ouch! I'd suggest you use git blame and we could track down
the offender, but I'm afraid it might be me so better to just
fix it and move on :)
Well, at least we should politely notify whoever reviewed that patch ...

Regards,
Robert


Faseela K
 

Jaya,
Check if this fixes the issue : https://git.opendaylight.org/gerrit/#/c/releng/builder/+/83781/
Thanks Thanh !
Thanks,
Faseela

-----Original Message-----
From: release <release-bounces@...> On Behalf Of Robert Varga
Sent: Wednesday, August 21, 2019 4:50 AM
To: Jamo Luhrsen <jluhrsen@...>; Jaya Priyadarshini <jaya.priyadarshini@...>; Abhijit Kumbhare <abhijitkoss@...>
Cc: netvirt-dev@...; Dayavanti Gopal Kamath <dayavanti.gopal.kamath@...>; integration-dev@...; Release <release@...>
Subject: Re: [release] [netvirt-dev] COE and Netvirt CSIT red

On 20/08/2019 21:29, Jamo Luhrsen wrote:
ouch! I'd suggest you use git blame and we could track down the
offender, but I'm afraid it might be me so better to just fix it and
move on :)
Well, at least we should politely notify whoever reviewed that patch ...

Regards,
Robert


JayaPr
 

Hi Faseela,

This doesn’t fix, fix is here:
https://git.opendaylight.org/gerrit/c/releng/builder/+/83836

Regards
Jaya

-----Original Message-----
From: Faseela K <faseela.k@...>
Sent: Wednesday, August 21, 2019 11:38 AM
To: Robert Varga <nite@...>; Jamo Luhrsen <jluhrsen@...>; Jaya Priyadarshini <jaya.priyadarshini@...>; Abhijit Kumbhare <abhijitkoss@...>
Cc: netvirt-dev@...; Dayavanti Gopal Kamath <dayavanti.gopal.kamath@...>; integration-dev@...; Release <release@...>
Subject: RE: [release] [netvirt-dev] COE and Netvirt CSIT red

Jaya,
Check if this fixes the issue : https://git.opendaylight.org/gerrit/#/c/releng/builder/+/83781/
Thanks Thanh !
Thanks,
Faseela

-----Original Message-----
From: release <release-bounces@...> On Behalf Of Robert Varga
Sent: Wednesday, August 21, 2019 4:50 AM
To: Jamo Luhrsen <jluhrsen@...>; Jaya Priyadarshini <jaya.priyadarshini@...>; Abhijit Kumbhare <abhijitkoss@...>
Cc: netvirt-dev@...; Dayavanti Gopal Kamath <dayavanti.gopal.kamath@...>; integration-dev@...; Release <release@...>
Subject: Re: [release] [netvirt-dev] COE and Netvirt CSIT red

On 20/08/2019 21:29, Jamo Luhrsen wrote:
ouch! I'd suggest you use git blame and we could track down the
offender, but I'm afraid it might be me so better to just fix it and
move on :)
Well, at least we should politely notify whoever reviewed that patch ...

Regards,
Robert


Faseela K
 

Thanks Jaya! Merged.

Thanks,
Faseela

-----Original Message-----
From: Jaya Priyadarshini <jaya.priyadarshini@...>
Sent: Wednesday, August 21, 2019 12:31 PM
To: Faseela K <faseela.k@...>; Robert Varga <nite@...>; Jamo Luhrsen <jluhrsen@...>; Abhijit Kumbhare <abhijitkoss@...>
Cc: netvirt-dev@...; Dayavanti Gopal Kamath <dayavanti.gopal.kamath@...>; integration-dev@...; Release <release@...>
Subject: RE: [release] [netvirt-dev] COE and Netvirt CSIT red

Hi Faseela,

This doesn’t fix, fix is here:
https://git.opendaylight.org/gerrit/c/releng/builder/+/83836

Regards
Jaya

-----Original Message-----
From: Faseela K <faseela.k@...>
Sent: Wednesday, August 21, 2019 11:38 AM
To: Robert Varga <nite@...>; Jamo Luhrsen <jluhrsen@...>; Jaya Priyadarshini <jaya.priyadarshini@...>; Abhijit Kumbhare <abhijitkoss@...>
Cc: netvirt-dev@...; Dayavanti Gopal Kamath <dayavanti.gopal.kamath@...>; integration-dev@...; Release <release@...>
Subject: RE: [release] [netvirt-dev] COE and Netvirt CSIT red

Jaya,
Check if this fixes the issue : https://git.opendaylight.org/gerrit/#/c/releng/builder/+/83781/
Thanks Thanh !
Thanks,
Faseela

-----Original Message-----
From: release <release-bounces@...> On Behalf Of Robert Varga
Sent: Wednesday, August 21, 2019 4:50 AM
To: Jamo Luhrsen <jluhrsen@...>; Jaya Priyadarshini <jaya.priyadarshini@...>; Abhijit Kumbhare <abhijitkoss@...>
Cc: netvirt-dev@...; Dayavanti Gopal Kamath <dayavanti.gopal.kamath@...>; integration-dev@...; Release <release@...>
Subject: Re: [release] [netvirt-dev] COE and Netvirt CSIT red

On 20/08/2019 21:29, Jamo Luhrsen wrote:
ouch! I'd suggest you use git blame and we could track down the
offender, but I'm afraid it might be me so better to just fix it and
move on :)
Well, at least we should politely notify whoever reviewed that patch ...

Regards,
Robert