Re: [controller-dev] [yangtools-dev] [releng] Distribution creation job

Luis Gomez <ecelgp@...>

Hi Vrakto,

Thanks for your mail, see my answers in-line.

On Feb 26, 2015, at 9:46 AM, Vratko Polak -X (vrpolak - Pantheon Technologies SRO at Cisco) <vrpolak@...> wrote:

Hi Luis.
As your TWS talk is nearing, I would like to present my views.
I think ${project}-integration job does basically the same thing
as ${project}-merge, only the triggers (and notifications) are different.
Build from source in ${project} git, run unit tests, upload artifacts.
*-merge is triggered by gerrit; *-integration should be triggered
by ${dependency}-merge and ${dependency}-integration jobs.
Maybe we should merge them to a single ${project}-publish job if possible.

Hi Vrakto, these jobs are releng responsibility (not integration), so the question is for Andy and Thahn :)

> Is there going to be a centralized 'this is what integration looks like' job?
Yes, in integration project, but it would generally be too slow
and prone to failures.
That is why there will be ${project}-distribution jobs,
which pack only subtree of features relevant to that project.
They will be triggered by ${project}-publish,
and they will be triggering ${project}-scit jobs with end-to-end tests.
The same tests will be run also on the full integration build,
but once again that job would be slower.
> So I merge stuff to yangtools
From point of view of BGPCEP project, the following would happen.
1. yangtools-merge triggers controller-integration

and ideally a yangtools-distribution job which builds integration karaf distribution

(and nothing else yet if Jenkins is good at dependency handling).
2. controller-integration triggers tcpmd5-integration (and unrelated jobs)

I do not think integration job calls another integration job, only a merge calls an integration

3. tcpmd5-integration finally triggers bgpcep-integration

bgpcep-integration should trigger directly from yangtools merge if you have declared the dependency

4. bgpcep-integration triggers bgpcep-distribution

This is not correct, bgpcep-distribution is only triggered on own merges.

5. bgpcep-distribution triggers bgpcep-csit jobs (several configurations).

bgpcep-csit jobs will be triggered after bgpcep-distribution yes but also after any <project>-distribution in the upstream like yangtools-distribution, this is why it is very important for projects in upstream to define their distribution job

6. bgpcep-csit-3nodes fails miserably, sends e-mails
to every upstream project. Probably including odlparent,
as Jenkins is not that good at back-tracing triggers.

So far I think it is only the bgpcep project that gets notified and the reason is I think it is better the project checks first why the system test is failing before it reports the issue to the upstream project, but we can change this if people think different.

96. Ultimately, integration-integration is triggered.
97. integration-integration triggers integration-distribution.
98. integration-distribution triggers integration-csit jobs.
99. After few hours all suites are finished (no idea who will be notified).

The projects will be the owners of the ODL system test so no plan to trigger anything in integration this release

Question to Luis:
Is Jenkins that good at dependency handling,
or would bgpcep-integration be triggered three times
(yangtools-merge, controller-integration and tcpmd5-integration)?

Should be only after merge

If no, that motivates the strategy of triggering from *-merge jobs only,
and that would create the artifact race issue Robert was talking about.
But we will have race issues anyway, as yangtools-merge
will not wait patiently for integration-distribution to finish
before merging the next yangtools change.

Although it is not the case here, according to Andy there is no issues in having 2 jobs publishing at the same time in Nexus as this last uses timestamps at millisecond precision

From: controller-dev-bounces@... [mailto:controller-dev-bounces@...]On Behalf Of Robert Varga
Sent: Wednesday, February 25, 2015 10:00 AM
To: Luis Gomez; yangtools-dev; opendaylight; openflowplugin-dev; openflowjava-dev; aaa-dev@...; vtn-dev@...; ovsdb-dev; l2switch-dev
Cc: dev (dev@...); 'integration-dev@...' (integration-dev@...) (integration-dev@...) (integration-dev@...)
Subject: Re: [controller-dev] [yangtools-dev] [releng] Distribution creation job

I am not sure I follow what exactly is the plan here. Is there going to be a centralized 'this is what integration looks like' job? Furthermore how does this interact with the ${project}-integration jobs, which also publish artifacts, but are not configured to trigger according to the dependency chain?

So I merge stuff to yangtools, which triggers a merge job, which will complete. Now two jobs will trigger: the controller integration job and this distribution job ... controller integration job publishes artifacts, so ... which artifacts is the distribution job going to pick up?


On 02/25/2015 03:20 AM, Luis Gomez wrote:
Hi all,
I am pushing following patches to releng repo:
This patch will create a job that downloads and builds the integration repository whenever there is a merge in the above projects. You can check the job template here:
You will need this job for 3 reasons:
1) To verify your changes are compatible with the integration code
2) To make sure your changes are immediately included in the integration distribution
3) To run some system test after the distribution is built
Please let me know if you have questions or concerns about merging this.
PS - these jobs will replace the old integration-polling jobs + integration-centralized-integration job

integration-dev mailing list

Join { to automatically receive all group messages.