All,
There was some breaking changes in CSIT/infra dependencies late last week that has put us on a path to upgrade (finally) to python3. There are multiple activities happening around this to bring CSIT back to a healthy state.
This is just FYI.
Please reach out if you have a specific issue that needs to be resolved and we can hopefully point you in the right direction. IRC still works, btw :)
Thanks, JamO
|
|
Re: CSIT jobs breaking on python2 trouble
I think its probably time for us to default to Python 3 considering Python 2 is EOL at this point. More and more packages will become Python 3 supported only so we'll likely to continue hitting issues in the future if we keep on Python 2.
Another patch I started looking into is consolidating all the Python requirements into a single requirements file ( https://git.opendaylight.org/gerrit/c/releng/builder/+/90435). One of the problems I find with Python dependencies is that when you run "pip install" for packages separately, Python's dependency manager does not take into account previously installed packages so later pip installs could pull in incompatible versions of similar dependencies thus breaking previous pip installed packages. Having all of the packages in a single requirements file and pip installing all of them together in 1 command will resolve this kind of issue.
I'm happy to help with a Python 3 migration but would need assistance from integration team to test such an effort.
Regards, Thanh
toggle quoted message
Show quoted text
On Fri, Jun 19, 2020 at 3:16 AM JamO Luhrsen < jluhrsen@...> wrote:
|
|
Re: CSIT jobs breaking on python2 trouble
|
|
Re: CSIT jobs breaking on python2 trouble
|
|
CSIT jobs breaking on python2 trouble
|
|
Re: builder[master]: Reduce Netvirt jobs costs
Srinivas <srinivas.rachakonda@...>
Hi Anil, Let me discuss with Team and get back to you. Thanks, Srinivas +91-9243478719
toggle quoted message
Show quoted text
From: integration-dev@... <integration-dev@...> On Behalf Of Anil Belur Sent: 18 June 2020 12:01 To: Jamo Luhrsen <jluhrsen@...> Cc: Luis Gomez <ecelgp@...>; Srinivas Rachakonda <srinivas.rachakonda@...>; app-dev@...; Bengt Thuree <bthuree@...>; integration-dev@... Subject: Re: [integration-dev] builder[master]: Reduce Netvirt jobs costs I think the Netvirt job count increased from ~400 to 570 with the table provided by Srinivas/Netvirt team, we need to make sure this reduces the frequency of jobs or there is very little scope for the job costs being saved here. On Thu, Jun 18, 2020 at 7:37 AM Jamo Luhrsen <jluhrsen@...> wrote: Yeah, this needs to be dialed in a little tighter. There is a lot that could be confusing. Some history: 1) there is a list of jobs that we skip in the daily AR run, like any tempest job, so in those got marked in the netvirt yaml with daily 2) we used to have a special high-frequency job that covered some netvirt jobs and ran multiple times a day. that was removed though: https://git.opendaylight.org/gerrit/c/releng/builder/+/81858 Srinivas, can you please dig through the definitions and remove the daily trigger from anything that is not in the list of jobs to skip. like here: https://github.com/opendaylight/releng-builder/blob/master/jjb/integration/integration-test-jobs.yaml#L65 If a job is in the list of jobs to skip (it's just matching on job name regex), then it will still need a trigger like daily or weekly. Good catch, Luis! JamO On 6/16/20 12:58 PM, Luis Gomez wrote: Guys, I just realized in this patch there is a new daily trigger: schedule: '{schedule-daily}' This has never been required before because the AR already runs daily and executes netvirt tests. So now I see the same test job runs twice a day (because AR + this new trigger):
On Jun 15, 2020, at 10:49 AM, Jamo Luhrsen <jluhrsen@...> wrote: ok, patch is merged. Thanks, JamO On 6/13/20 8:26 PM, Srinivas Rachakonda wrote: Hi Jamo, I have reviewed the scripts and it looks fine. On Sat, 13 Jun 2020 at 01:49, Jamo Luhrsen <jluhrsen@...> wrote: Srinivas, Can you please check this patch to see if it's in line with what you were expecting? Thanks, JamO #netvirt On 6/11/20 1:13 AM, Gerrit Code Review wrote: Patch set 2:Code-Review +1 View Change To view, visit change 90385. To unsubscribe, or for help writing mail filters, visit settings. Gerrit-Project: releng/builder Gerrit-Change-Id: Ice7702c633e8f49b2b02f0828d130bb48f762e81 Gerrit-Change-Number: 90385 Gerrit-Comment-Date: Thu, 11 Jun 2020 08:13:11 +0000 Gerrit-MessageType: comment
|
|
Re: builder[master]: Reduce Netvirt jobs costs
Hi Srinivas:
I think the Netvirt job count increased from ~400 to 570 with the table provided by Srinivas/Netvirt team, we need to make sure this reduces the frequency of jobs or there is very little scope for the job costs being saved here.
Thanks, Anil
toggle quoted message
Show quoted text
On Thu, Jun 18, 2020 at 7:37 AM Jamo Luhrsen < jluhrsen@...> wrote:
Yeah, this needs to be dialed in a little tighter. There is a lot
that could be
confusing. Some history:
1) there is a list of jobs that we skip in the daily AR run, like
any tempest
job, so in those got marked in the netvirt yaml with daily
2) we used to have a special high-frequency job that covered some
netvirt
jobs and ran multiple times a day. that was removed though:
https://git.opendaylight.org/gerrit/c/releng/builder/+/81858
Srinivas, can you please dig through the definitions and remove
the daily
trigger from anything that is not in the list of jobs to skip.
like here:
https://github.com/opendaylight/releng-builder/blob/master/jjb/integration/integration-test-jobs.yaml#L65
If a job is in the list of jobs to skip (it's just matching on
job name regex),
then it will still need a trigger like daily or weekly.
Good catch, Luis!
JamO
On 6/16/20 12:58 PM, Luis Gomez wrote:
Guys, I just realized in this patch there is a new daily trigger:
schedule: '{schedule-daily}'
This has never been required before because the AR
already runs daily and executes netvirt tests.
So now I see the same test job runs twice a day
(because AR + this new trigger):
Is this intended or not?
BR/Luis
On Jun 15, 2020, at 10:49 AM, Jamo Luhrsen
< jluhrsen@...> wrote:
ok, patch is merged.
Thanks,
JamO
On 6/13/20 8:26 PM,
Srinivas Rachakonda wrote:
Hi Jamo,
I have reviewed the scripts and it
looks fine.
Marked +1 in the review.
R.Srinivas
+91-9243478719
On Sat, 13 Jun
2020 at 01:49, Jamo Luhrsen < jluhrsen@...>
wrote:
Srinivas,
Can you please check this patch to
see if it's in line with what you were
expecting?
Thanks,
JamO
#netvirt
On 6/11/20 1:13 AM, Gerrit Code
Review wrote:
Patch set 2:Code-Review
+1
View Change
To view, visit change 90385. To unsubscribe,
or for help writing mail filters, visit settings.
Gerrit-Project: releng/builder
Gerrit-Branch: master
Gerrit-Change-Id:
Ice7702c633e8f49b2b02f0828d130bb48f762e81
Gerrit-Change-Number: 90385
Gerrit-PatchSet: 2
Gerrit-Comment-Date: Thu, 11 Jun 2020
08:13:11 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment
|
|
Re: builder[master]: Reduce Netvirt jobs costs
Srinivas <srinivas.rachakonda@...>
toggle quoted message
Show quoted text
From: integration-dev@... <integration-dev@...> On Behalf Of JamO Luhrsen Sent: 18 June 2020 03:07 To: Luis Gomez <ecelgp@...>; Anil Belur <abelur@...> Cc: Srinivas Rachakonda <srinivas.rachakonda@...>; app-dev@...; bthuree@...; integration-dev@... Subject: Re: [integration-dev] builder[master]: Reduce Netvirt jobs costs Yeah, this needs to be dialed in a little tighter. There is a lot that could be confusing. Some history: 1) there is a list of jobs that we skip in the daily AR run, like any tempest job, so in those got marked in the netvirt yaml with daily 2) we used to have a special high-frequency job that covered some netvirt jobs and ran multiple times a day. that was removed though: https://git.opendaylight.org/gerrit/c/releng/builder/+/81858 Srinivas, can you please dig through the definitions and remove the daily trigger from anything that is not in the list of jobs to skip. like here: https://github.com/opendaylight/releng-builder/blob/master/jjb/integration/integration-test-jobs.yaml#L65 If a job is in the list of jobs to skip (it's just matching on job name regex), then it will still need a trigger like daily or weekly. Good catch, Luis! JamO On 6/16/20 12:58 PM, Luis Gomez wrote: Guys, I just realized in this patch there is a new daily trigger: schedule: '{schedule-daily}' This has never been required before because the AR already runs daily and executes netvirt tests. So now I see the same test job runs twice a day (because AR + this new trigger):
On Jun 15, 2020, at 10:49 AM, Jamo Luhrsen <jluhrsen@...> wrote: ok, patch is merged. Thanks, JamO On 6/13/20 8:26 PM, Srinivas Rachakonda wrote: Hi Jamo, I have reviewed the scripts and it looks fine. On Sat, 13 Jun 2020 at 01:49, Jamo Luhrsen <jluhrsen@...> wrote: Srinivas, Can you please check this patch to see if it's in line with what you were expecting? Thanks, JamO #netvirt On 6/11/20 1:13 AM, Gerrit Code Review wrote: Patch set 2:Code-Review +1 View Change To view, visit change 90385. To unsubscribe, or for help writing mail filters, visit settings. Gerrit-Project: releng/builder Gerrit-Change-Id: Ice7702c633e8f49b2b02f0828d130bb48f762e81 Gerrit-Change-Number: 90385 Gerrit-Comment-Date: Thu, 11 Jun 2020 08:13:11 +0000 Gerrit-MessageType: comment
|
|
Re: builder[master]: Reduce Netvirt jobs costs
Yeah, this needs to be dialed in a little tighter. There is a lot
that could be
confusing. Some history:
1) there is a list of jobs that we skip in the daily AR run, like
any tempest
job, so in those got marked in the netvirt yaml with daily
2) we used to have a special high-frequency job that covered some
netvirt
jobs and ran multiple times a day. that was removed though:
https://git.opendaylight.org/gerrit/c/releng/builder/+/81858
Srinivas, can you please dig through the definitions and remove
the daily
trigger from anything that is not in the list of jobs to skip.
like here:
https://github.com/opendaylight/releng-builder/blob/master/jjb/integration/integration-test-jobs.yaml#L65
If a job is in the list of jobs to skip (it's just matching on
job name regex),
then it will still need a trigger like daily or weekly.
Good catch, Luis!
JamO
On 6/16/20 12:58 PM, Luis Gomez wrote:
toggle quoted message
Show quoted text
Guys, I just realized in this patch there is a new daily trigger:
schedule: '{schedule-daily}'
This has never been required before because the AR
already runs daily and executes netvirt tests.
So now I see the same test job runs twice a day
(because AR + this new trigger):
Is this intended or not?
BR/Luis
On Jun 15, 2020, at 10:49 AM, Jamo Luhrsen
< jluhrsen@...> wrote:
ok, patch is merged.
Thanks,
JamO
On 6/13/20 8:26 PM,
Srinivas Rachakonda wrote:
Hi Jamo,
I have reviewed the scripts and it
looks fine.
Marked +1 in the review.
R.Srinivas
+91-9243478719
On Sat, 13 Jun
2020 at 01:49, Jamo Luhrsen < jluhrsen@...>
wrote:
Srinivas,
Can you please check this patch to
see if it's in line with what you were
expecting?
Thanks,
JamO
#netvirt
On 6/11/20 1:13 AM, Gerrit Code
Review wrote:
Patch set 2:Code-Review
+1
View Change
To view, visit change 90385. To unsubscribe,
or for help writing mail filters, visit settings.
Gerrit-Project: releng/builder
Gerrit-Branch: master
Gerrit-Change-Id:
Ice7702c633e8f49b2b02f0828d130bb48f762e81
Gerrit-Change-Number: 90385
Gerrit-PatchSet: 2
Gerrit-Comment-Date: Thu, 11 Jun 2020
08:13:11 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment
|
|
no integration meeting tomorrow (06/16/2020)
I cannot make our normal 9pm meeting time, so let's cancel this week.
Thanks, JamO
|
|
Re: Netvirt jobs are failing with stack error: uwsgi directory no found
Thanks. I gave a review just now.
JamO
toggle quoted message
Show quoted text
Hi Jamo,
Raised the below review:
https://git.opendaylight.org/gerrit/c/releng/builder/+/90459
Thanks,
Srinivas
+91-9243478719
The last passing job is installing uWSGI v2.0.18 so maybe
v2.0.19 is broken? Maybe you can
try pinning it back to v2.0.18 to see if it helps?
Thanks,
JamO
Hi Jamo/Anil,
Netvirt CSIT jobs are failing with the
below error since morning.
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upgrade-snat-conntrack-magnesium/274/control_1/stack.log.gz
2020-06-15
10:54:16.154 | Downloading https://files.pythonhosted.org/packages/6f/e9/729f0eb15f4847c729c1a841bcbb5e327d08d208de0cbe661224955c9297/uWSGI-2.0.19.tar.gz
(804kB)
2020-06-15
10:54:17.017 | Saved ./uWSGI-2.0.19.tar.gz
2020-06-15
10:54:19.063 | Successfully downloaded uwsgi
2020-06-15
10:54:19.808 | You are using pip version 9.0.3, however
version 20.1.1 is available.
2020-06-15
10:54:19.808 | You should consider upgrading via the 'pip
install --upgrade pip' command.
2020-06-15
10:54:19.853 | + lib/apache:install_apache_uwsgi:97
: local uwsgi
2020-06-15
10:54:19.858 | ++ lib/apache:install_apache_uwsgi:98
: ls 'uwsgi*'
2020-06-15
10:54:19.861 | ls: cannot access uwsgi*: No such file or
directory
2020-06-15
10:54:19.865 | + lib/apache:install_apache_uwsgi:98
: uwsgi=
2020-06-15
10:54:19.869 | + lib/apache:install_apache_uwsgi:1
: exit_trap
2020-06-15
10:54:19.873 | + ./stack.sh:exit_trap:516
: local r=2
2020-06-15
10:54:19.877 | ++ ./stack.sh:exit_trap:517
: jobs -p
2020-06-15
10:54:19.879 | + ./stack.sh:exit_trap:517
: jobs=
2020-06-15
10:54:19.881 | + ./stack.sh:exit_trap:520
: [[ -n '' ]]
2020-06-15
10:54:19.883 | + ./stack.sh:exit_trap:526
: '[' -f '' ']'
2020-06-15
10:54:19.885 | + ./stack.sh:exit_trap:531
: kill_spinner
2020-06-15
10:54:19.887 | + ./stack.sh:kill_spinner:430
: '[' '!' -z '' ']'
2020-06-15
10:54:19.889 | + ./stack.sh:exit_trap:533
: [[ 2 -ne 0 ]]
2020-06-15
10:54:19.891 | + ./stack.sh:exit_trap:534
: echo 'Error on exit'
2020-06-15
10:54:19.891 | Error on exit
2020-06-15
10:54:19.894 | + ./stack.sh:exit_trap:536
: type -p generate-subunit
2020-06-15
10:54:19.896 | + ./stack.sh:exit_trap:537
: generate-subunit 1592218203 256 fail
2020-06-15
10:54:21.128 | + ./stack.sh:exit_trap:539
: [[ -z /opt/stack/logs ]]
2020-06-15
10:54:21.132 | + ./stack.sh:exit_trap:542
: /opt/stack/devstack/tools/worlddump.py -d
/opt/stack/logs
2020-06-15
10:54:21.513 | ovs-vsctl:
unix:/var/run/openvswitch/db.sock: database connection
failed (No such file or directory)
2020-06-15
10:54:21.616 | ovs-vsctl:
unix:/var/run/openvswitch/db.sock: database connection
failed (No such file or directory)
2020-06-15
10:54:22.462 | + ./stack.sh:exit_trap:551
: exit 2
Could you please look into the issue.
Thanks,
Srinivas
+91-9243478719
|
|
Re: Netvirt jobs are failing with stack error: uwsgi directory no found
Srinivas <srinivas.rachakonda@...>
toggle quoted message
Show quoted text
From: Jamo Luhrsen <jluhrsen@...> Sent: 16 June 2020 00:04 To: srinivas.rachakonda@...; 'Anil Belur' <abelur@...> Cc: 'Thanh Ha' <zxiiro@...>; 'Dayavanti Gopal Kamath' <dayavanti.gopal.kamath@...>; 'P S Ravikanth' <ravikanth.ps@...>; 'odl netvirt dev' <netvirt-dev@...>; 'Karthikeyan Krishnan' <karthikeyan.k@...>; integration-dev@... Subject: Re: Netvirt jobs are failing with stack error: uwsgi directory no found The last passing job is installing uWSGI v2.0.18 so maybe v2.0.19 is broken? Maybe you can try pinning it back to v2.0.18 to see if it helps? Thanks, JamO Hi Jamo/Anil, Netvirt CSIT jobs are failing with the below error since morning. https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upgrade-snat-conntrack-magnesium/274/control_1/stack.log.gz 2020-06-15 10:54:16.154 | Downloading https://files.pythonhosted.org/packages/6f/e9/729f0eb15f4847c729c1a841bcbb5e327d08d208de0cbe661224955c9297/uWSGI-2.0.19.tar.gz (804kB) 2020-06-15 10:54:17.017 | Saved ./uWSGI-2.0.19.tar.gz 2020-06-15 10:54:19.063 | Successfully downloaded uwsgi 2020-06-15 10:54:19.808 | You are using pip version 9.0.3, however version 20.1.1 is available. 2020-06-15 10:54:19.808 | You should consider upgrading via the 'pip install --upgrade pip' command. 2020-06-15 10:54:19.853 | + lib/apache:install_apache_uwsgi:97 : local uwsgi 2020-06-15 10:54:19.858 | ++ lib/apache:install_apache_uwsgi:98 : ls 'uwsgi*' 2020-06-15 10:54:19.861 | ls: cannot access uwsgi*: No such file or directory 2020-06-15 10:54:19.865 | + lib/apache:install_apache_uwsgi:98 : uwsgi= 2020-06-15 10:54:19.869 | + lib/apache:install_apache_uwsgi:1 : exit_trap 2020-06-15 10:54:19.873 | + ./stack.sh:exit_trap:516 : local r=2 2020-06-15 10:54:19.877 | ++ ./stack.sh:exit_trap:517 : jobs -p 2020-06-15 10:54:19.879 | + ./stack.sh:exit_trap:517 : jobs= 2020-06-15 10:54:19.881 | + ./stack.sh:exit_trap:520 : [[ -n '' ]] 2020-06-15 10:54:19.883 | + ./stack.sh:exit_trap:526 : '[' -f '' ']' 2020-06-15 10:54:19.885 | + ./stack.sh:exit_trap:531 : kill_spinner 2020-06-15 10:54:19.887 | + ./stack.sh:kill_spinner:430 : '[' '!' -z '' ']' 2020-06-15 10:54:19.889 | + ./stack.sh:exit_trap:533 : [[ 2 -ne 0 ]] 2020-06-15 10:54:19.891 | + ./stack.sh:exit_trap:534 : echo 'Error on exit' 2020-06-15 10:54:19.891 | Error on exit 2020-06-15 10:54:19.894 | + ./stack.sh:exit_trap:536 : type -p generate-subunit 2020-06-15 10:54:19.896 | + ./stack.sh:exit_trap:537 : generate-subunit 1592218203 256 fail 2020-06-15 10:54:21.128 | + ./stack.sh:exit_trap:539 : [[ -z /opt/stack/logs ]] 2020-06-15 10:54:21.132 | + ./stack.sh:exit_trap:542 : /opt/stack/devstack/tools/worlddump.py -d /opt/stack/logs 2020-06-15 10:54:21.513 | ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) 2020-06-15 10:54:21.616 | ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory) 2020-06-15 10:54:22.462 | + ./stack.sh:exit_trap:551 : exit 2 Could you please look into the issue. Thanks, Srinivas +91-9243478719
|
|
Re: Netvirt jobs are failing with stack error: uwsgi directory no found
The last passing job is installing uWSGI v2.0.18 so maybe v2.0.19
is broken? Maybe you can
try pinning it back to v2.0.18 to see if it helps?
Thanks,
JamO
toggle quoted message
Show quoted text
Hi Jamo/Anil,
Netvirt CSIT jobs are failing with the
below error since morning.
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upgrade-snat-conntrack-magnesium/274/control_1/stack.log.gz
2020-06-15
10:54:16.154 | Downloading
https://files.pythonhosted.org/packages/6f/e9/729f0eb15f4847c729c1a841bcbb5e327d08d208de0cbe661224955c9297/uWSGI-2.0.19.tar.gz
(804kB)
2020-06-15
10:54:17.017 | Saved ./uWSGI-2.0.19.tar.gz
2020-06-15
10:54:19.063 | Successfully downloaded uwsgi
2020-06-15
10:54:19.808 | You are using pip version 9.0.3, however
version 20.1.1 is available.
2020-06-15
10:54:19.808 | You should consider upgrading via the 'pip
install --upgrade pip' command.
2020-06-15
10:54:19.853 | + lib/apache:install_apache_uwsgi:97
: local uwsgi
2020-06-15
10:54:19.858 | ++ lib/apache:install_apache_uwsgi:98
: ls 'uwsgi*'
2020-06-15
10:54:19.861 | ls: cannot access uwsgi*: No such file or
directory
2020-06-15
10:54:19.865 | + lib/apache:install_apache_uwsgi:98
: uwsgi=
2020-06-15
10:54:19.869 | + lib/apache:install_apache_uwsgi:1
: exit_trap
2020-06-15
10:54:19.873 | + ./stack.sh:exit_trap:516
: local r=2
2020-06-15
10:54:19.877 | ++ ./stack.sh:exit_trap:517
: jobs -p
2020-06-15
10:54:19.879 | + ./stack.sh:exit_trap:517
: jobs=
2020-06-15
10:54:19.881 | + ./stack.sh:exit_trap:520
: [[ -n '' ]]
2020-06-15
10:54:19.883 | + ./stack.sh:exit_trap:526
: '[' -f '' ']'
2020-06-15
10:54:19.885 | + ./stack.sh:exit_trap:531
: kill_spinner
2020-06-15
10:54:19.887 | + ./stack.sh:kill_spinner:430
: '[' '!' -z '' ']'
2020-06-15
10:54:19.889 | + ./stack.sh:exit_trap:533
: [[ 2 -ne 0 ]]
2020-06-15
10:54:19.891 | + ./stack.sh:exit_trap:534
: echo 'Error on exit'
2020-06-15
10:54:19.891 | Error on exit
2020-06-15
10:54:19.894 | + ./stack.sh:exit_trap:536
: type -p generate-subunit
2020-06-15
10:54:19.896 | + ./stack.sh:exit_trap:537
: generate-subunit 1592218203 256 fail
2020-06-15
10:54:21.128 | + ./stack.sh:exit_trap:539
: [[ -z /opt/stack/logs ]]
2020-06-15
10:54:21.132 | + ./stack.sh:exit_trap:542
: /opt/stack/devstack/tools/worlddump.py -d
/opt/stack/logs
2020-06-15
10:54:21.513 | ovs-vsctl: unix:/var/run/openvswitch/db.sock:
database connection failed (No such file or directory)
2020-06-15
10:54:21.616 | ovs-vsctl: unix:/var/run/openvswitch/db.sock:
database connection failed (No such file or directory)
2020-06-15
10:54:22.462 | + ./stack.sh:exit_trap:551
: exit 2
Could you please look into the issue.
Thanks,
Srinivas
+91-9243478719
|
|
Re: [release] controller CSIT for Aluminium
On Fri, Jun 12, 2020 at 2:02 PM Robert Varga < nite@...> wrote: On 12/06/2020 19:52, Robert Varga wrote:
> On 09/06/2020 02:49, JamO Luhrsen wrote:
>> Hi all,
>>
>> I've had an action item for a while to dig through Aluminium controller
>> CSIT to
>> try to get a handle on where we stand after the MRI bumps. This has turned
>> up some jira tickets:
>
> [snip]
>
>> Trouble on the test side:
>>
>> https://jira.opendaylight.org/browse/INTTEST-100
>
> This is across all releases, I just updated the ticket and raised it to
> blocker, as it does block any attempt to release anything. I have
> attached known-good and known-bad console.log and the 'diff -U 0'
> between them.
>
> It seems to be pointing at
> https://git.opendaylight.org/gerrit/c/releng/builder/+/90196/2/jjb/integration/integration-templates.yaml
>
> which would seem to indicate trouble with python setup in the new image.
>
> Thanh, any idea on how to debug this further?
This patch https://git.opendaylight.org/gerrit/c/releng/builder/+/90433 will at least make mininet available to be used on the sandbox directly. Maybe we can run a few commands and try to reproduce on the sandbox with a stripped down job that just tries to run the script. I'll work with JamO to see if we can find anything useful to debug.
Regards, Thanh
|
|
Re: [release] controller CSIT for Aluminium
On 12/06/2020 19:52, Robert Varga wrote: On 09/06/2020 02:49, JamO Luhrsen wrote:
Hi all,
I've had an action item for a while to dig through Aluminium controller CSIT to try to get a handle on where we stand after the MRI bumps. This has turned up some jira tickets: [snip]
Trouble on the test side:
https://jira.opendaylight.org/browse/INTTEST-100 This is across all releases, I just updated the ticket and raised it to blocker, as it does block any attempt to release anything. I have attached known-good and known-bad console.log and the 'diff -U 0' between them.
It seems to be pointing at https://git.opendaylight.org/gerrit/c/releng/builder/+/90196/2/jjb/integration/integration-templates.yaml
which would seem to indicate trouble with python setup in the new image.
Thanh, any idea on how to debug this further? +correct email Thanks, Robert
|
|
Re: [release] controller CSIT for Aluminium
On 09/06/2020 02:49, JamO Luhrsen wrote: Hi all,
I've had an action item for a while to dig through Aluminium controller CSIT to try to get a handle on where we stand after the MRI bumps. This has turned up some jira tickets: [snip] Trouble on the test side:
https://jira.opendaylight.org/browse/INTTEST-100 This is across all releases, I just updated the ticket and raised it to blocker, as it does block any attempt to release anything. I have attached known-good and known-bad console.log and the 'diff -U 0' between them. It seems to be pointing at https://git.opendaylight.org/gerrit/c/releng/builder/+/90196/2/jjb/integration/integration-templates.yamlwhich would seem to indicate trouble with python setup in the new image. Thanh, any idea on how to debug this further? Thanks, Robert
|
|
Re: validate-autorelease-aluminium failing
The check conditions have changed after we introduced MRI projects, so now it is possible to have projects in distribution but not in AR. Because of this, we can either rework the script to check something different (not sure what) or just remove as I did.
toggle quoted message
Show quoted text
On Jun 12, 2020, at 10:06 AM, Thanh Ha < zxiiro@...> wrote:
This just turns off the comparison entirely so not really fixing but I think it's the right approach until someone can fix the script. Did a quick look at the script and it looks like it parses pom.xml files dynamically to figure out what to compare which means we definitely need to adjust the script to allow exceptions for MRI projects.
Regards, Thanh On Fri, Jun 12, 2020 at 12:22 PM Luis Gomez < ecelgp@...> wrote: This should fix: https://git.opendaylight.org/gerrit/c/releng/builder/+/90432On Jun 12, 2020, at 9:13 AM, Robert Varga < nite@...> wrote:
Well, we fixed feature packaging in distribution (i.e. we now include features-controller-testing and others). Also, validate-autorelease triggers only when pom.xmls are modified I think, hence it is not typically seen. Regards, Robert On 12/06/2020 18:11, Luis Gomez wrote: I think you are right, but why this starts to fail now?
On Jun 11, 2020, at 11:41 PM, Robert Varga <nite@...> wrote:
Hello,
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/infrautils-validate-autorelease-aluminium/2/console.log.gz failed with:
[infrautils-validate-autorelease-aluminium] $ /bin/bash -l /tmp/jenkins5280574077411919720.sh ---> compare-autorelease-projects.sh ERROR: List of projects in integration/distribution but NOT in releng/autorelease: mdsal yangtools controller Build step 'Execute shell' marked build as failure
I think this is wrong, since MRI projects need to be in the distribution and not be present in autorelease.
Perhaps compare-autorelease-projects.sh needs to be updated with list of MRI projects?
Regards, Robert
|
|
Re: validate-autorelease-aluminium failing
This just turns off the comparison entirely so not really fixing but I think it's the right approach until someone can fix the script. Did a quick look at the script and it looks like it parses pom.xml files dynamically to figure out what to compare which means we definitely need to adjust the script to allow exceptions for MRI projects.
Regards, Thanh
toggle quoted message
Show quoted text
On Fri, Jun 12, 2020 at 12:22 PM Luis Gomez < ecelgp@...> wrote: This should fix: https://git.opendaylight.org/gerrit/c/releng/builder/+/90432On Jun 12, 2020, at 9:13 AM, Robert Varga < nite@...> wrote:
Well, we fixed feature packaging in distribution (i.e. we now include features-controller-testing and others). Also, validate-autorelease triggers only when pom.xmls are modified I think, hence it is not typically seen. Regards, Robert On 12/06/2020 18:11, Luis Gomez wrote: I think you are right, but why this starts to fail now?
On Jun 11, 2020, at 11:41 PM, Robert Varga <nite@...> wrote:
Hello,
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/infrautils-validate-autorelease-aluminium/2/console.log.gz failed with:
[infrautils-validate-autorelease-aluminium] $ /bin/bash -l /tmp/jenkins5280574077411919720.sh ---> compare-autorelease-projects.sh ERROR: List of projects in integration/distribution but NOT in releng/autorelease: mdsal yangtools controller Build step 'Execute shell' marked build as failure
I think this is wrong, since MRI projects need to be in the distribution and not be present in autorelease.
Perhaps compare-autorelease-projects.sh needs to be updated with list of MRI projects?
Regards, Robert
|
|
Re: validate-autorelease-aluminium failing
toggle quoted message
Show quoted text
On Jun 12, 2020, at 9:13 AM, Robert Varga < nite@...> wrote:
Well, we fixed feature packaging in distribution (i.e. we now include features-controller-testing and others). Also, validate-autorelease triggers only when pom.xmls are modified I think, hence it is not typically seen. Regards, Robert On 12/06/2020 18:11, Luis Gomez wrote: I think you are right, but why this starts to fail now?
On Jun 11, 2020, at 11:41 PM, Robert Varga <nite@...> wrote:
Hello,
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/infrautils-validate-autorelease-aluminium/2/console.log.gz failed with:
[infrautils-validate-autorelease-aluminium] $ /bin/bash -l /tmp/jenkins5280574077411919720.sh ---> compare-autorelease-projects.sh ERROR: List of projects in integration/distribution but NOT in releng/autorelease: mdsal yangtools controller Build step 'Execute shell' marked build as failure
I think this is wrong, since MRI projects need to be in the distribution and not be present in autorelease.
Perhaps compare-autorelease-projects.sh needs to be updated with list of MRI projects?
Regards, Robert
|
|
Re: validate-autorelease-aluminium failing
Well, we fixed feature packaging in distribution (i.e. we now include features-controller-testing and others).
Also, validate-autorelease triggers only when pom.xmls are modified I think, hence it is not typically seen.
Regards, Robert
toggle quoted message
Show quoted text
On 12/06/2020 18:11, Luis Gomez wrote: I think you are right, but why this starts to fail now?
On Jun 11, 2020, at 11:41 PM, Robert Varga <nite@...> wrote:
Hello,
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/infrautils-validate-autorelease-aluminium/2/console.log.gz failed with:
[infrautils-validate-autorelease-aluminium] $ /bin/bash -l /tmp/jenkins5280574077411919720.sh ---> compare-autorelease-projects.sh ERROR: List of projects in integration/distribution but NOT in releng/autorelease: mdsal yangtools controller Build step 'Execute shell' marked build as failure I think this is wrong, since MRI projects need to be in the distribution and not be present in autorelease.
Perhaps compare-autorelease-projects.sh needs to be updated with list of MRI projects?
Regards, Robert
|
|