Creating a cleaner distribution-test job


Jamo Luhrsen <jluhrsen@...>
 

To All Projects,

and especially to:

AAA
BGPCEP
CAPWAP
Controller
L2Switch
NetConf
NIC
OpenFlowPlugin
OVSDB
SXP
VPNService


Moving forward we want to attempt to maintain a clean distribution-test job [0],
which is our jenkins project that runs many of our system test (CSIT) jobs. Some
details about this job:

- This is run with every distribution autorelease produces.
- It is the job we use as the Integration team to discern the quality of release
candidates.
- Can be used to test gerrit patches, using the "test-integration" keyword [1]

We also envision downstream consumers (e.g. OPNFV) being able to quickly check these
results when determining if they can consume a nightly distribution or if it may be
broken in some way. This can be a valuable piece toward moving to faster release
cycles, as well.

In order to maximize the benefit of this job, we need all CSIT jobs running
there to adhere to our system test case expectations [2].

So, in the short term we have removed [3][4] all projects' (see list above
for affected projects) CSIT jobs that are not meeting these expectations.
Having CSIT jobs running there is a release requirement [5] in order to
avoid features being labeled as experimental, so although the current number
of CSIT jobs may be low in our Boron distribution test job, I expect that
projects will start to bring their CSIT jobs to a more reliable state and
then introduce them back (or for the first time) via this job .yaml [6]

Please reach out to the Integration team if you have any questions.

Thanks,
JamO

[0] https://jenkins.opendaylight.org/releng/view/Distribution-Jobs/job/integration-distribution-test-beryllium/
[1] https://opendaylight.readthedocs.io/en/latest/submodules/releng/builder/docs/jenkins.html#patch-test-job
[2] https://wiki.opendaylight.org/view/Integration/Test/Test_Case_Expectations
[3] https://git.opendaylight.org/gerrit/#/c/39453/
[4] https://git.opendaylight.org/gerrit/#/c/39451
[5] https://wiki.opendaylight.org/view/CrossProject:Integration_Group:Boron_Traditional_Release_Project_Integration_Requirements#system_test
[6] https://github.com/opendaylight/releng-builder/blob/master/jjb/integration/integration-test-jobs.yaml