Completely agree here, though the closer we get to Helium release, the less value a stable release will have.
Historically, by the time we cut the Hydrogen release, we knew that there will need to be a follow-up, service release. Vast majority (if not all) of developers who contributed patches to yangtools, controller, bgpcep and openflowjava were operating on the assumption that master is in bugfix-only mode, awaiting the decision when such a release will be cut. That decision has taken two months to materialize and when it did, the mode of its execution created major hunk of work -- so now the same body of developers are expected to re-live two months of their time (and we are past 3 month mark now).
Ed's proposal, which was voiced during the last hackfest, allows for both things. It quickly creates a branch which can be stabilized. It also unblocks master for receiving shiny new stuff.
We do know what patches should go in (at least from the perspective when we started the effort, that has drifted already, as master has continued to receive bugfixes! ). The point of the proposal is to execute quickly. Attempting to follow the 'replay the history' approach has cost us huge amounts of both developer and real time -- time which could have been invested in hardening the system.
From my perspective we have three options here:
1) perform a limited stable release now, with the explicit knowledge it is not useful for a lot of projects
2) execute on Ed's plan, which will give us a steady state within a week
3) continue executing the current approach, which will give us a steady state some time before mid-June (judging based on current progress)
Bye,
Robert
toggle quoted message
Show quoted text
On 05/06/2014 09:41 PM, Omar Sultan (osultan) wrote: So, I think the road to software hell is paved with such intentions, that we will fix things in the next release. My experience is the shiny new stuff gets more attention then fixing and documenting the old stuff.
If we have accumulated some technical debt, it would prefer we clear it and have something clean and stable that folks can use for a basis for Helium. Otherwise, it seems to me like we are starting work on the second story of house when the walls are still wobbly on the first floor.
O
Sent from my iPad
On May 6, 2014, at 12:21 PM, "Moiz Raja (moraja)" <moraja@...> wrote:
An honest question. Why do we need to do a stable release for Hydrogen? Wouldn't it be just easier to get to Helium and thereafter make sure that we start doing stable releases. It seems like we're using more energy in creating the stable release than what it's worth. AFAIK Hydrogen itself was a developer-only release (or atleast that's what it turned out to be) so we do not have any customers waiting for our fixes.
my 20 cents,
-Moiz
________________________________________
From: discuss-bounces@... [discuss-bounces@...] on behalf of Ed Warnicke (eaw)
Sent: Tuesday, May 06, 2014 10:22 AM
To: Chris Wright
Cc: <discuss@...>; tsc@...
Subject: Re: [OpenDaylight Discuss] [OpenDaylight TSC] Hydrogen Stable Release Proposal
On May 5, 2014, at 10:55 PM, Chris Wright <chrisw@...> wrote:
* Ed Warnicke (eaw) (eaw@...) wrote:
Trying to replay the sequences of patches, across all of our (12 I believe) projects with various
interdependencies between them has proven to be a herculean task, particularly in light of global
timezones and Jenkins verifies/merges. Lots of projects are getting wedged on other projects
as they attempt to cherry pick down historical patches to a relatively new stable branch. Please be more specific. What issues, which projects, where are the
bugs...details… Sure… controller has been blocking almost daily on some issues or other related to
yangtools (either it played forward to fast, or it needs to catch up).
I’ve talked to the openflowplugin guys who are having blocking issues with synchronization
with controller.
Add timezone delays and Jenkins verify and merge delays… and everything slows to a crawl.
Mostly folks have been busy trying to solve these issues as they arise… making progress not reporting it...
It appears to me that the root issue is attempting to replay the past.
To resolve this I would suggest that we adopt the following tact:
1) Projects (if they so desire) pull their stable branches from a current point in time (today,or
if it takes us a while to discuss this a little later this week).
2) Projects update their versions to append the -1-SNAPSHOT (we’ve done this already in our first attempt
at this).
3) Projects revert the patches inappropriate for a stable branch (I think most projects have done the triage on this,
so its just a bit of doing, and since most projects have put almost all of their effort on master into bug fixing, is a much
smaller number of things).
4) Going forward we follow Chris’s recommendation to commit *new* bug fixes first to master and then (if appropriate)
to stable (such double committing should be done contemporaneously so we don’t fall back into needing to replay the past).
This should allow us to get unwedged on the ongoing ‘replay the past’ problem and get down to testing and hardening
the stable release.
Thoughts? I think you risk trading one problem for another and in a few weeks back
to where we are now.
I don’t think so… the fundamental problem is replaying the past, and the fact that we are
playing back a large number of bug fixes from the past to avoid a small number of non-bug fixes. And that introduces (unsurprisingly
when you think about it) a large number of ordering issues. At current rate of process, I suspect we would take
months of stop start stop start stop start trying to replay the past… with this approach I suspect we could
pull today, and revert the few patches that need reverting over the course of a week or so and get onto focusing
on hardening the release.
Think of it this way… we have a time ordered stream of patches, some of which depend on each other.
Those time ordered streams get distributed across our 12 projects in Hydrogen. This goes on for hundreds of patches,
all but tens of them bug fixes appropriate for stable.
Then we pull the stable branch and try to put humpty dumpty back together again. We still have hundreds of patches
to replay, in the correct dependency order, across 12 projects… and tens to exclude. Some projects get ahead, and thus
other projects can’t verify. Some projects are behind, and thus other projects must then coordinate and block waiting for them
to catch up.
When you look at what is actually implied by the current approach (the full problems of which were not immediately obvious to me until we
started hitting them)… I’m actually amazed we aren’t more blocked than we are.
The proposal is quite simple… come to a state we know is currently working (now), and revert the tens of patches that are not appropriate
to stable and proceed with the double commit strategy for things that need to go into stable going forward. Much smaller surface area to create difficulty.
Much fewer issues. From down here in the trenches its a much smaller chance of hitting these kinds of issues again.
Its basically putting us on the footing we would have been if we adopted the original plan *at the time of Hydrogen Release* instead of months later
when we had hundreds of patches worth of past to figure out how to replay in the right order.
To date there are exactly two stated issues:
1) bgpcep relying on config-manager change in controller as captured in Bug 903.
2) ovsdb relying on some unspecified change in controller There have been way more than this… but as mentioned above, folks have been focused
on solving those problems rather than reporting them. Rather than filing bugs and reporting status
folks have been working on solving the problems and getting unblocked.
This is not "lots" it's only two. So either there are "lots" in which
case we need real details (and we have a very fundamental issue that
projects are not speaking up), or it's only two in which case we may
be able to work through those two issues. Again, when you are closer to the code, and have your hands deeper in getting folks unblocked… you see
*lots*.
Ed
_______________________________________________
Discuss mailing list
Discuss@...
https://lists.opendaylight.org/mailman/listinfo/discuss
_______________________________________________
Discuss mailing list
Discuss@...
https://lists.opendaylight.org/mailman/listinfo/discuss _______________________________________________
TSC mailing list
TSC@...
https://lists.opendaylight.org/mailman/listinfo/tsc
|
