builder[master]: Reduce Netvirt jobs costs


Srinivas <srinivas.rachakonda@...>
 

Hi Anil,

 

Let me discuss with Team and get back to you.

 

Thanks,

Srinivas

+91-9243478719

 

From: integration-dev@... <integration-dev@...> On Behalf Of Anil Belur
Sent: 18 June 2020 12:01
To: Jamo Luhrsen <jluhrsen@...>
Cc: Luis Gomez <ecelgp@...>; Srinivas Rachakonda <srinivas.rachakonda@...>; app-dev@...; Bengt Thuree <bthuree@...>; integration-dev@...
Subject: Re: [integration-dev] builder[master]: Reduce Netvirt jobs costs

 

Hi Srinivas: 

 

I think the Netvirt job count increased from ~400 to 570 with the table provided by Srinivas/Netvirt team, we need to make sure this reduces the frequency of jobs or there is very little scope for the job costs being saved here. 

 

Thanks,

Anil

 

On Thu, Jun 18, 2020 at 7:37 AM Jamo Luhrsen <jluhrsen@...> wrote:

Yeah, this needs to be dialed in a little tighter. There is a lot that could be
confusing. Some history:

1) there is a list of jobs that we skip in the daily AR run, like any tempest
    job, so in those got marked in the netvirt yaml with daily

2) we used to have a special high-frequency job that covered some netvirt
    jobs and ran multiple times a day. that was removed though:
    https://git.opendaylight.org/gerrit/c/releng/builder/+/81858

 

Srinivas, can you please dig through the definitions and remove the daily
trigger from anything that is not in the list of jobs to skip. like here:

https://github.com/opendaylight/releng-builder/blob/master/jjb/integration/integration-test-jobs.yaml#L65

If a job is in the list of jobs to skip (it's just matching on job name regex),
then it will still need a trigger like daily or weekly.

 

Good catch, Luis!

 

JamO

 

 

On 6/16/20 12:58 PM, Luis Gomez wrote:

Guys, I just realized in this patch there is a new daily trigger:

 

schedule: '{schedule-daily}'

 

This has never been required before because the AR already runs daily and executes netvirt tests.

 

So now I see the same test job runs twice a day (because AR + this new trigger):

 

 

Is this intended or not?

 

BR/Luis

 



On Jun 15, 2020, at 10:49 AM, Jamo Luhrsen <jluhrsen@...> wrote:

 

ok, patch is merged.

Thanks,
JamO

On 6/13/20 8:26 PM, Srinivas Rachakonda wrote:

Hi Jamo,

 

I have reviewed the scripts and it looks fine.

Marked +1 in the review.

 

R.Srinivas

+91-9243478719

 

 

 

On Sat, 13 Jun 2020 at 01:49, Jamo Luhrsen <jluhrsen@...> wrote:

Srinivas,

Can you please check this patch to see if it's in line with what you were expecting?

Thanks,
JamO

#netvirt

On 6/11/20 1:13 AM, Gerrit Code Review wrote:

Patch set 2:Code-Review +1

View Change

To view, visit change 90385. To unsubscribe, or for help writing mail filters, visit settings.

Gerrit-Project: releng/builder

Gerrit-Branch: master

Gerrit-Change-Id: Ice7702c633e8f49b2b02f0828d130bb48f762e81

Gerrit-Change-Number: 90385

Gerrit-PatchSet: 2

Gerrit-Owner: Anil Belur <abelur@...>

Gerrit-Reviewer: Bengt Thuree <bthuree@...>

Gerrit-Reviewer: Jamo Luhrsen <jluhrsen@...>

Gerrit-Reviewer: Luis Gomez <ecelgp@...>

Gerrit-Reviewer: Srinivas Rachakonda <srinivas.rachakonda@...>

Gerrit-Reviewer: Thanh Ha (zxiiro) <zxiiro@...>

Gerrit-Reviewer: jenkins-releng <jenkins-releng@...>

Gerrit-Comment-Date: Thu, 11 Jun 2020 08:13:11 +0000

Gerrit-HasComments: No

Gerrit-Has-Labels: Yes

Gerrit-MessageType: comment

 


Anil Belur
 

Hi Srinivas: 

I think the Netvirt job count increased from ~400 to 570 with the table provided by Srinivas/Netvirt team, we need to make sure this reduces the frequency of jobs or there is very little scope for the job costs being saved here. 

Thanks,
Anil


On Thu, Jun 18, 2020 at 7:37 AM Jamo Luhrsen <jluhrsen@...> wrote:

Yeah, this needs to be dialed in a little tighter. There is a lot that could be
confusing. Some history:

1) there is a list of jobs that we skip in the daily AR run, like any tempest
    job, so in those got marked in the netvirt yaml with daily

2) we used to have a special high-frequency job that covered some netvirt
    jobs and ran multiple times a day. that was removed though:
    https://git.opendaylight.org/gerrit/c/releng/builder/+/81858


Srinivas, can you please dig through the definitions and remove the daily
trigger from anything that is not in the list of jobs to skip. like here:

https://github.com/opendaylight/releng-builder/blob/master/jjb/integration/integration-test-jobs.yaml#L65

If a job is in the list of jobs to skip (it's just matching on job name regex),
then it will still need a trigger like daily or weekly.


Good catch, Luis!


JamO



On 6/16/20 12:58 PM, Luis Gomez wrote:
Guys, I just realized in this patch there is a new daily trigger:

schedule: '{schedule-daily}'

This has never been required before because the AR already runs daily and executes netvirt tests.

So now I see the same test job runs twice a day (because AR + this new trigger):


Is this intended or not?

BR/Luis


On Jun 15, 2020, at 10:49 AM, Jamo Luhrsen <jluhrsen@...> wrote:

ok, patch is merged.

Thanks,
JamO

On 6/13/20 8:26 PM, Srinivas Rachakonda wrote:
Hi Jamo,

I have reviewed the scripts and it looks fine.
Marked +1 in the review.

R.Srinivas
+91-9243478719



On Sat, 13 Jun 2020 at 01:49, Jamo Luhrsen <jluhrsen@...> wrote:

Srinivas,

Can you please check this patch to see if it's in line with what you were expecting?

Thanks,
JamO

#netvirt

On 6/11/20 1:13 AM, Gerrit Code Review wrote:

Patch set 2:Code-Review +1

View Change

To view, visit change 90385. To unsubscribe, or for help writing mail filters, visit settings.

Gerrit-Project: releng/builder
Gerrit-Branch: master
Gerrit-Change-Id: Ice7702c633e8f49b2b02f0828d130bb48f762e81
Gerrit-Change-Number: 90385
Gerrit-PatchSet: 2
Gerrit-Owner: Anil Belur <abelur@...>
Gerrit-Reviewer: Bengt Thuree <bthuree@...>
Gerrit-Reviewer: Jamo Luhrsen <jluhrsen@...>
Gerrit-Reviewer: Luis Gomez <ecelgp@...>
Gerrit-Reviewer: Srinivas Rachakonda <srinivas.rachakonda@...>
Gerrit-Reviewer: Thanh Ha (zxiiro) <zxiiro@...>
Gerrit-Reviewer: jenkins-releng <jenkins-releng@...>
Gerrit-Comment-Date: Thu, 11 Jun 2020 08:13:11 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment


Srinivas <srinivas.rachakonda@...>
 

Hi Jamo,

 

Raised the below review:

https://git.opendaylight.org/gerrit/c/releng/builder/+/90493

 

 

Thanks,

Srinivas

+91-9243478719

 

From: integration-dev@... <integration-dev@...> On Behalf Of JamO Luhrsen
Sent: 18 June 2020 03:07
To: Luis Gomez <ecelgp@...>; Anil Belur <abelur@...>
Cc: Srinivas Rachakonda <srinivas.rachakonda@...>; app-dev@...; bthuree@...; integration-dev@...
Subject: Re: [integration-dev] builder[master]: Reduce Netvirt jobs costs

 

Yeah, this needs to be dialed in a little tighter. There is a lot that could be
confusing. Some history:

1) there is a list of jobs that we skip in the daily AR run, like any tempest
    job, so in those got marked in the netvirt yaml with daily

2) we used to have a special high-frequency job that covered some netvirt
    jobs and ran multiple times a day. that was removed though:
    https://git.opendaylight.org/gerrit/c/releng/builder/+/81858

 

Srinivas, can you please dig through the definitions and remove the daily
trigger from anything that is not in the list of jobs to skip. like here:

https://github.com/opendaylight/releng-builder/blob/master/jjb/integration/integration-test-jobs.yaml#L65

If a job is in the list of jobs to skip (it's just matching on job name regex),
then it will still need a trigger like daily or weekly.

 

Good catch, Luis!

 

JamO

 

 

On 6/16/20 12:58 PM, Luis Gomez wrote:

Guys, I just realized in this patch there is a new daily trigger:

 

schedule: '{schedule-daily}'

 

This has never been required before because the AR already runs daily and executes netvirt tests.

 

So now I see the same test job runs twice a day (because AR + this new trigger):

 

 

Is this intended or not?

 

BR/Luis

 



On Jun 15, 2020, at 10:49 AM, Jamo Luhrsen <jluhrsen@...> wrote:

 

ok, patch is merged.

Thanks,
JamO

On 6/13/20 8:26 PM, Srinivas Rachakonda wrote:

Hi Jamo,

 

I have reviewed the scripts and it looks fine.

Marked +1 in the review.

 

R.Srinivas

+91-9243478719

 

 

 

On Sat, 13 Jun 2020 at 01:49, Jamo Luhrsen <jluhrsen@...> wrote:

Srinivas,

Can you please check this patch to see if it's in line with what you were expecting?

Thanks,
JamO

#netvirt

On 6/11/20 1:13 AM, Gerrit Code Review wrote:

Patch set 2:Code-Review +1

View Change

To view, visit change 90385. To unsubscribe, or for help writing mail filters, visit settings.

Gerrit-Project: releng/builder

Gerrit-Branch: master

Gerrit-Change-Id: Ice7702c633e8f49b2b02f0828d130bb48f762e81

Gerrit-Change-Number: 90385

Gerrit-PatchSet: 2

Gerrit-Owner: Anil Belur <abelur@...>

Gerrit-Reviewer: Bengt Thuree <bthuree@...>

Gerrit-Reviewer: Jamo Luhrsen <jluhrsen@...>

Gerrit-Reviewer: Luis Gomez <ecelgp@...>

Gerrit-Reviewer: Srinivas Rachakonda <srinivas.rachakonda@...>

Gerrit-Reviewer: Thanh Ha (zxiiro) <zxiiro@...>

Gerrit-Reviewer: jenkins-releng <jenkins-releng@...>

Gerrit-Comment-Date: Thu, 11 Jun 2020 08:13:11 +0000

Gerrit-HasComments: No

Gerrit-Has-Labels: Yes

Gerrit-MessageType: comment

 


JamO Luhrsen
 

Yeah, this needs to be dialed in a little tighter. There is a lot that could be
confusing. Some history:

1) there is a list of jobs that we skip in the daily AR run, like any tempest
    job, so in those got marked in the netvirt yaml with daily

2) we used to have a special high-frequency job that covered some netvirt
    jobs and ran multiple times a day. that was removed though:
    https://git.opendaylight.org/gerrit/c/releng/builder/+/81858


Srinivas, can you please dig through the definitions and remove the daily
trigger from anything that is not in the list of jobs to skip. like here:

https://github.com/opendaylight/releng-builder/blob/master/jjb/integration/integration-test-jobs.yaml#L65

If a job is in the list of jobs to skip (it's just matching on job name regex),
then it will still need a trigger like daily or weekly.


Good catch, Luis!


JamO



On 6/16/20 12:58 PM, Luis Gomez wrote:

Guys, I just realized in this patch there is a new daily trigger:

schedule: '{schedule-daily}'

This has never been required before because the AR already runs daily and executes netvirt tests.

So now I see the same test job runs twice a day (because AR + this new trigger):


Is this intended or not?

BR/Luis


On Jun 15, 2020, at 10:49 AM, Jamo Luhrsen <jluhrsen@...> wrote:

ok, patch is merged.

Thanks,
JamO

On 6/13/20 8:26 PM, Srinivas Rachakonda wrote:
Hi Jamo,

I have reviewed the scripts and it looks fine.
Marked +1 in the review.

R.Srinivas
+91-9243478719



On Sat, 13 Jun 2020 at 01:49, Jamo Luhrsen <jluhrsen@...> wrote:

Srinivas,

Can you please check this patch to see if it's in line with what you were expecting?

Thanks,
JamO

#netvirt

On 6/11/20 1:13 AM, Gerrit Code Review wrote:

Patch set 2:Code-Review +1

View Change

To view, visit change 90385. To unsubscribe, or for help writing mail filters, visit settings.

Gerrit-Project: releng/builder
Gerrit-Branch: master
Gerrit-Change-Id: Ice7702c633e8f49b2b02f0828d130bb48f762e81
Gerrit-Change-Number: 90385
Gerrit-PatchSet: 2
Gerrit-Owner: Anil Belur <abelur@...>
Gerrit-Reviewer: Bengt Thuree <bthuree@...>
Gerrit-Reviewer: Jamo Luhrsen <jluhrsen@...>
Gerrit-Reviewer: Luis Gomez <ecelgp@...>
Gerrit-Reviewer: Srinivas Rachakonda <srinivas.rachakonda@...>
Gerrit-Reviewer: Thanh Ha (zxiiro) <zxiiro@...>
Gerrit-Reviewer: jenkins-releng <jenkins-releng@...>
Gerrit-Comment-Date: Thu, 11 Jun 2020 08:13:11 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment