Re: Invitation: OpenDaylight Release Mgmt & Release Engineering Needs @ Wed Apr 23, 2014 11am - 12pm (probb at

Rochelle.RochelleGrober <rochelle.grober@...>

Just wanted to post the minutes of the meeting as Huawei tried to capture them.  We will have a contingent at the OpenStack summit and hope to meet some of the ODL folks at a BoF if we can arrange it.




4/23/14   OpenDayLight meeting on CI and Release issues and how Huawei can help



Ed Warnicke (Cisco) -- rabble-rouser

Chris Wright (RedHat, ODL Board Member ) – release process under control his goal.  Is a developer from virtualization and Linux, now SDN

Phil  Robb (Director of Networking Solutions, Linux Foundation) – Core to Open DayLight.

Luis  (Brocade) – internal distribution, continuous integation ,  needs/want quality standards

Leena(?)(Cisco?) --   working on integration, release, and maintaining stable branches

Giles (?) missed his area of focus

Madhu (?)  lots of projects – complains on lack of management for scalability

Mark Brenner (Sweitzer Engineering Labs) – end user

Manish – end user

Vaishali(?)( Cisco) – controller project  interested in versioning and CI, CR, working on automated release with  Shurehka(?)

Peter Posar (Ericsson)

Ed?  Also on line

Shuo Yang (Huawei)

Yue Chen (Huawei)

Jerry Zhao (Huawei)

Rochelle Grober (Huawei)

*Andrew Grimberg (cc’ed but not in attendance) is an important player in ODL -- wants to disaggregate nexus and Jenkins silos



·         There are currently about 250 projects that are actively part of ODL. OpenStack has 21 official projects that all gate in the CI process and 62 projects meeting every week and 265 repositories in Stackforge (we have 3 for Compass and there are lots of infrastructure ones as well).  So, they are young and either very siloed or very broad in their project directions.

·         Build process is different from OpenStack in that it uses Maven (with POM files mostly replacing OpenStack requirements files) and a repository broker/cache agent that sits between Git and Jenkins.  Gerrit is still used, but testing is thin and lots of breakage happens across projects, some of which is traceable to Maven architectural issues, some of which is lack of automated enforcement of rules.

·         First major release is Hydrogen, which was just released.  The project is attempting Simultaneous Releases but has problems with “cat herding”.  Helium is due out 9/29 and Lithium planning has started.

·         Simultaneous release is voluntary and must be signed up for – Participation requires some QA and sync standards.  The group is still working out the standards through discussions, hence the issue of “cat herding”

·         The team has decided that for projects volunteering for Simultaneous Release, there needs to be a project contact specifically to provide communication between the release team and the project team.  For Hydrogen, they manage to get negative (problem) status reporting from the contacts, and are now also getting positive status reports.  Status includes whether some other project broke theirs, or they broke some other project, or (best) everything is still working with the current set of checkins.  The contact person has responsibility to limit/reduce breakage.

·         Most of the concerns expressed in the meeting was about needing more automation, not making the process as heavy as OpenStack’s (to keep up velocity), more QA and testing, and more management (as in herding cats management).  It appears that a “real” infra/release team doesn’t exist yet, but it is starting to coalesce.

·         The ODL folks communicated a number of process formalizations that have been or are beginning to happen:

o   Milestones every 4 weeks

o   Well publicized cutoff date for new features/projects in a release

o   Mailing lists for each project, plus the main one (discuss) and build related ones

o   Lots and lots of IRCs

o   A plan to have a support plan for releases drafted by Milestone4

o   A release team planning IRC meeting 1 week before each milestone

o   A Continuous Release project up and available tomorrow

o   Active work on improving the Wiki for new and existing Developers

o   Documentation consolidated into a single team


Outcome on Huawei side:  Jerry Zhao and Rochelle Grober to join ODL and participate in Integration and release process definition/improvement/automation.


Action Item:  ODL OpenStack summit attendee(s): would be great to schedule an ODL BoF at the summit.  Huawei will have folks there to meet everyone F2F.

Join to automatically receive all group messages.