Jenkins flooded by global-jjb-rtd-merge jobs


Thanh ha <zxiiro@...>
 

Hi Everyone,

We noticed (thanks JamO) today that it appears global-jjb-rtd-merge jobs are flooding the ODL Jenkins system with excessive amounts of jobs.

It turns out that this is caused by the ref-update trigger which appears to pick up all comments as triggers for this job so every time there's discussions or comments in a review it causes Jenkins to queue an additional job.

This is likely because newer versions of Gerrit stores comment data in the git repo using NoteDB which likely stores the data in a separate branch under refs/* in Git's database (I'm not 100% sure on this but this is my current guess based on what I know).

In any case I opened a support ticket with LF to find a resolution to this flooding of jobs.


Regards,
Thanh


JamO Luhrsen
 

Thanks Thanh...

On 1/23/20 11:25 AM, Thanh ha wrote:
Hi Everyone,

We noticed (thanks JamO) today that it appears global-jjb-rtd-merge jobs are flooding the ODL Jenkins system with excessive amounts of jobs.

seems like more than 100 jobs triggered in the past 24h. Each job is only running for a few minutes,
but they seem to mostly fail (didn't look why).

JamO

It turns out that this is caused by the ref-update trigger which appears to pick up all comments as triggers for this job so every time there's discussions or comments in a review it causes Jenkins to queue an additional job.

This is likely because newer versions of Gerrit stores comment data in the git repo using NoteDB which likely stores the data in a separate branch under refs/* in Git's database (I'm not 100% sure on this but this is my current guess based on what I know).

In any case I opened a support ticket with LF to find a resolution to this flooding of jobs.


Regards,
Thanh



    


Thanh ha <zxiiro@...>
 

On Thu, 23 Jan 2020 at 14:25, Thanh Ha <zxiiro@...> wrote:
Hi Everyone,

We noticed (thanks JamO) today that it appears global-jjb-rtd-merge jobs are flooding the ODL Jenkins system with excessive amounts of jobs.

It turns out that this is caused by the ref-update trigger which appears to pick up all comments as triggers for this job so every time there's discussions or comments in a review it causes Jenkins to queue an additional job.

This is likely because newer versions of Gerrit stores comment data in the git repo using NoteDB which likely stores the data in a separate branch under refs/* in Git's database (I'm not 100% sure on this but this is my current guess based on what I know).

In any case I opened a support ticket with LF to find a resolution to this flooding of jobs.


Hi Anil,

It's been 2 weeks and we haven't received any response from the LF on this issue. Can you find out what's holding this one up?

Thanks,
Thanh