1. Integration-Dev
  2. Topics

Saving infra resources

Luis Gomez
 

I think this patch should do the work for now:


BR/Luis

On Jan 18, 2019, at 10:27 AM, Luis Gomez <ecelgp@...> wrote:

See below

On Jan 17, 2019, at 8:09 PM, Anil Belur <abelur@...> wrote:

Luis:

With the `trigger-parameterized-builds` macro/plugin does not have a switch which will automagically trigger jobs only if there are none in the queue. The way around this to use the manual way (without the plugin) to trigger builds and build some logic over checking if the jobs are in the queue before triggering. However, once a job is already in the queue (it already been triggered to start with some data from the older AR builds, which is probably outdated), and therefore should remove existing int/dist job from the queue before trigger adds a new int/dist job.


Most popular:

1) Use Jenkins groovy script to clear job in queue.
2) Pass the same parameters to the distribution test job, this means the test job should be able fetch the BUNDLE_URL (only thing that changes between builds) from some place.

  

I'd think this should be fixed in AR/parent job that is triggering these int/dist jobs every day. The question that needs to be answered is if we do need to build AR (and int/dist jobs) every day even if there are no new changes to the project participating in the AR repos. 

AR job does not consume much resources compared to distribution test job + we want to be able to detect regression at least in a daily basis so I would not change current AR trigger.


If there is scope for some improvements, I think we should be fix this in both places. 

Thanks,
Anil     

On Fri, Jan 18, 2019 at 4:36 AM Luis Gomez <ecelgp@...> wrote:
Hi,

I have observed this a few times now (see attached pic):

<Screen Shot 2019-01-17 at 10.25.49 AM.png>


The distribution test jobs pile up in Jenkins queue and every one of these takes a lot of infra resources (full distribution CSIT verification). Is there a way for Jenkins to only keep 1 job of this type in the queue? like a new job will remove an old one if this exists in the queue?

BR/Luis
_______________________________________________
integration-dev mailing list
integration-dev@...
https://lists.opendaylight.org/mailman/listinfo/integration-dev
Luis Gomez
 

See below

On Jan 17, 2019, at 8:09 PM, Anil Belur <abelur@...> wrote:

Luis:

With the `trigger-parameterized-builds` macro/plugin does not have a switch which will automagically trigger jobs only if there are none in the queue. The way around this to use the manual way (without the plugin) to trigger builds and build some logic over checking if the jobs are in the queue before triggering. However, once a job is already in the queue (it already been triggered to start with some data from the older AR builds, which is probably outdated), and therefore should remove existing int/dist job from the queue before trigger adds a new int/dist job.


Most popular:

1) Use Jenkins groovy script to clear job in queue.
2) Pass the same parameters to the distribution test job, this means the test job should be able fetch the BUNDLE_URL (only thing that changes between builds) from some place.

  

I'd think this should be fixed in AR/parent job that is triggering these int/dist jobs every day. The question that needs to be answered is if we do need to build AR (and int/dist jobs) every day even if there are no new changes to the project participating in the AR repos. 

AR job does not consume much resources compared to distribution test job + we want to be able to detect regression at least in a daily basis so I would not change current AR trigger.


If there is scope for some improvements, I think we should be fix this in both places. 

Thanks,
Anil     

On Fri, Jan 18, 2019 at 4:36 AM Luis Gomez <ecelgp@...> wrote:
Hi,

I have observed this a few times now (see attached pic):

<Screen Shot 2019-01-17 at 10.25.49 AM.png>


The distribution test jobs pile up in Jenkins queue and every one of these takes a lot of infra resources (full distribution CSIT verification). Is there a way for Jenkins to only keep 1 job of this type in the queue? like a new job will remove an old one if this exists in the queue?

BR/Luis
_______________________________________________
integration-dev mailing list
integration-dev@...
https://lists.opendaylight.org/mailman/listinfo/integration-dev
Abhijit Kumbhare
 

About the stuff below - to address Robert's point about quiescent period and xtend breaking us - maybe reduce the frequency from daily to weekly (or some other suitable interval like twice a week)?

On Fri, Jan 18, 2019 at 2:11 AM Robert Varga <nite@...> wrote:
On 18/01/2019 05:09, Anil Belur wrote:

> I'd think this should be fixed in AR/parent job that is triggering these
> int/dist jobs every day. The question that needs to be answered is if we
> do need to build AR (and int/dist jobs) every day even if there are no
> new changes to the project participating in the AR repos.

Since we do not have fully-reproducible builds we cannot assume the
build does not need to trigger just because the source code has not
changed -- one example would be a quiescent period and xtend breaking us :)

Regards,
Robert
Robert Varga
 

On 18/01/2019 05:09, Anil Belur wrote:
Luis:

With the `trigger-parameterized-builds` macro/plugin does not have a
switch which will automagically trigger jobs only if there are none in
the queue. The way around this to use the manual way (without the
plugin) to trigger builds and build some logic over checking if the jobs
are in the queue before triggering. However, once a job is already in
the queue (it already been triggered to start with some data from the
older AR builds, which is probably outdated), and therefore should
remove existing int/dist job from the queue before trigger adds a
new int/dist job. 
Note this applies to merge jobs as well, but there we have a rather
different requirement: the pre-existing enqueued-but-not-started job
should never be canceled.

I'd think this should be fixed in AR/parent job that is triggering these
int/dist jobs every day. The question that needs to be answered is if we
do need to build AR (and int/dist jobs) every day even if there are no
new changes to the project participating in the AR repos.
Since we do not have fully-reproducible builds we cannot assume the
build does not need to trigger just because the source code has not
changed -- one example would be a quiescent period and xtend breaking us :)

Regards,
Robert

signature.asc

signature.asc

Anil Belur
 

Luis:

With the `trigger-parameterized-builds` macro/plugin does not have a switch which will automagically trigger jobs only if there are none in the queue. The way around this to use the manual way (without the plugin) to trigger builds and build some logic over checking if the jobs are in the queue before triggering. However, once a job is already in the queue (it already been triggered to start with some data from the older AR builds, which is probably outdated), and therefore should remove existing int/dist job from the queue before trigger adds a new int/dist job.  

I'd think this should be fixed in AR/parent job that is triggering these int/dist jobs every day. The question that needs to be answered is if we do need to build AR (and int/dist jobs) every day even if there are no new changes to the project participating in the AR repos. 

If there is scope for some improvements, I think we should be fix this in both places. 

Thanks,
Anil     

On Fri, Jan 18, 2019 at 4:36 AM Luis Gomez <ecelgp@...> wrote:
Hi,

I have observed this a few times now (see attached pic):



The distribution test jobs pile up in Jenkins queue and every one of these takes a lot of infra resources (full distribution CSIT verification). Is there a way for Jenkins to only keep 1 job of this type in the queue? like a new job will remove an old one if this exists in the queue?

BR/Luis
_______________________________________________
integration-dev mailing list
integration-dev@...
https://lists.opendaylight.org/mailman/listinfo/integration-dev
Abhijit Kumbhare
 

Thanks Luis for coming up with ideas on this topic.


On Thu, Jan 17, 2019 at 10:36 AM Luis Gomez <ecelgp@...> wrote:
Hi,

I have observed this a few times now (see attached pic):



The distribution test jobs pile up in Jenkins queue and every one of these takes a lot of infra resources (full distribution CSIT verification). Is there a way for Jenkins to only keep 1 job of this type in the queue? like a new job will remove an old one if this exists in the queue?

BR/Luis
_______________________________________________
integration-dev mailing list
integration-dev@...
https://lists.opendaylight.org/mailman/listinfo/integration-dev
Luis Gomez
 

Hi,

I have observed this a few times now (see attached pic):



The distribution test jobs pile up in Jenkins queue and every one of these takes a lot of infra resources (full distribution CSIT verification). Is there a way for Jenkins to only keep 1 job of this type in the queue? like a new job will remove an old one if this exists in the queue?

BR/Luis

1 - 7 of 7