my perspective on where we are with the single controller code base issue
|
David Meyer <dmm@...>
All,
As you know today (05.14.2013) was the day on which I wanted to have the single controller code base issue resolved. What I had said is that if we couldn't get the projects to agree on a way forward that we as a TSC would vote on the issue. In the mean time we have reached general consensus that the Dixon-Erickson OpenDaylight Merged Controller Proposal (DE) (see https://wiki.opendaylight.org/view/Dixon-Erickson_OpenDaylight_Merged_Controller_Proposal). I want to be very clear here that that plan is distinct from what is being called Proposed Dixon-Erickson Execution Plan (see https://wiki.opendaylight.org/view/Proposed_Dixon-Erickson_Execution_Plan). So from this perspective the community has come together over this general direction. This functionally obviates the need for a direct vote. There have been quite a few concerns raised both publicly and to me directly in private email. I'd like to review those here and provide my perspective. (i). The SAL It is generally agreed that there is work left to be done on the SAL, and that there may be instances in which, (in the near term) we need a mechanism to provide a kind of "SAL bypass". Chris' IOCTL proposal is one such idea. What I want to point out here is the the ODP is evolving and as such we expect the architecture to not not improve technically but converge on the needed functionality/generality over time. This is a completely natural state of affairs for any software project. The summary is that we can provide short term mechanisms that bridge any perceived or real functionality gaps in the SAL (again, Chris' IOCTL idea might be one such mechanism). So while the OSCP folks have argued pretty strongly against the SAL (both publicly and privately), the DE proposal endorses the SAL concept and the OpenDaylight Controller folks consider the SAL to be a fundamental architectural construct. That said, evolution of the SAL (be it the model driven approach, bypass, etc) is an issue for folks who are writing SAL applications (committing code) to resolve. (ii). New Project/Repository This issue is really about the fairness of commits to the controller code base. This was stated explicitly to me privately and publicly by Glenn Ricart and Guru Parulkar on the discuss list. On the other hand, a simple majority of TSC members (and many in the community) have expressed the opinion that they would prefer moving forward with the plan outlined in the DE-proposal, namely start with the OpenDaylight controller and evolve that as needed. I will note here that there is no objective evidence that there is a fairness issue with commits to any candiate code base (it would hard for that to be the case in any event as we haven't had the chance to really get going). Finally, since this is largely a political issue it is outside of the scope of the TSC and should be "policed" by the community. As an aside, I will also note here that several community members have expressed concern that the new project approach is quite expensive and has little benefit beyond the political/perception issue mentioned above. (iii) Timing The timing issue, namely that there is (is not) a rush to get the ODP project up and running is one that has also been raised both publicly and privately. I have been told (again, both publicly and privately) by the OSCP (BSN) folks that they have time pressure to port their applications to the ODP controller. That is a principle rationale behind the Layered API proposal (https://wiki.opendaylight.org/view/Layered_API_Merged_Controller_Proposal). In addition, the same team is also arguing that we are in no rush and in fact any date for resolving the controller issue is not only artificial but damaging to the community; this exact sentiment was contained in a private communication from Rob Sherwood to me dated Tue, May 14, 2013 at 1:38 AM. In the spirit of transparency I am copying the relevant text from that email here: "I know that in your heart you believe what you're doing is the right thing for the community, and that's why I've held off making a stink, but as chair, I'm not sure what you're doing is in terms of pushing hard on this artificial deadline (that you said originally was not set in stone but now it is) or being opaque in terms of the actual process is really helping the community." -- Rob Sherwood to dmm on Tue, May 14, 2013 at 1:38 AM. (this text of course also contains criticisms of me in my role as chair to which I will return but I want to focus here on the timing issue). The summary here is that it has been argued by the OSCP team that we accept their proposal (https://wiki.opendaylight.org/view/File:LayeredAPIProposal.pdf) because there is time pressure to port applications while at the same time it is being argued that any time frame is damaging, artificial, or worse. On the other hand, a simple majority of the TSC have stated publicly (on the discuss list) that resolving the controller code base issue of primary importance and they would like to move forward with an interpretation of DE that starts with the existing OpenDaylight controller code base. It is clear from these inconsistencies that the timing issue is largely a non-issue for the ODP community at large; in addition there is clear benefit to moving forward with the DE approach now. One might also note here that we've made very little progress over the last many weeks and it is time to start, now. (iv). Finally, if anyone has problems with me personally and/or the way TSC is being run please air those publicly on the discuss list. I will not be responding to emails such as the one quoted above, lobbying by private phone call, or the like. Again, if you have complaints about the way the TSC is being run please discuss those on the discuss or tsc lists. Summary: It is clear that there is additional evolution that is needed for the SAL. No one is arguing otherwise. The new project/repository issue is really about concern that commits won't be fair (again, both Glenn and Guru raised this); however there is no evidence that this is or will be the case and in any event is not a technical issue and as such is out of scope for the TSC. Finally, the timing issue is largely a commercial issue (the need to port applications) and also has been argued on both sides by the same people and as such I consider those arguments largely moot; there is a real need to get the consortium going (hence the date) but that is a wholly different argument. --dmm |
|
|
|
David Meyer <dmm@...>
BTW, it has been pointed out that I mis-attributed the IOCTL idea to
toggle quoted message
Show quoted text
Chris. That was my mistake. That said, the point I was making still holds, namely, that there exist some ideas (and there will be more) to deal with the near term deficiencies, perceived or otherwise, with the SAL. --dmm On Tue, May 14, 2013 at 9:09 AM, David Meyer <dmm@...> wrote:
All, |
|
|
|
Kyle Forster
Dave (and other members of the community) - The folks that work here at BSN are here to make Daylight as successful as we can, but i) we owe transparency about our over-arching concern (that we think is valid for a large swath of the community), ii) we'd like to go to a few facts and iii) we'd like to chart a productive path forward. ------- Transparency -- The over-arching concern that we (BSN folk) have tried to mitigate with design and process proposals is a "tyranny of the committer majority" problem in the controller project, a situation where a highly concentrated group of committers blocks well intentioned progress by teams around them whose applications call for different design trade-offs and timing priorities. This issue is very real in other well known SDN open source projects where competing companies are regularly locked out from material commits. The result is an awkward amalgamation of open and closed source components across a handful of companies now, a trail of open source that isn't broadly used in production and a lot of confusion. The problem isn't because the concentrated group of committers in these projects have been evil, or dumb, or mean. They simply prioritize designs for the needs they see and add friction for the needs they don't until those needs go away. While subjective, I submit to the list that the same patterns are emerging in Daylight already, and it will take extraordinary effort by motivated individuals in the months ahead to avoid this path. ------- Let's go to facts -- First, the D-E plan you linked to above does *not* specify a starting repo in writing, but rather states explicitly on the page that the authors disagree on this point, a point that has been re-emphasized in the public threads. D-E plan, full steam ahead! Which D-E plan starting repo? Second, on community support, by my count of emails on list, I came up with 10 organizations voicing support for a fresh repo / committer list and 3 against. Have we asked for on-list support from others who share our concerns privately? Yes! But even if we filter by TSC organizations (community+$$$), we came up with 3 in favor, 3 against, 3 no explicit vote. -------- How to move forward? -- We are in strong favor of the fresh repo and blank committer list with Dave and Colin as the first two committers. Let them invite other committers to the "merged controller repo" along with their code as it flows to their design. This is the suggestion I believe that both Dave and Colin have supported as their own compromise as the authors of the D-E plan. It has the nice property of parallelizing work -- as we've drilled down, there is a lot to do on SAL to make it ready for production-grade Floodlight components, and a lot to do on making production-grade Floodlight components SAL-ready. We're signed up for a lot there. It has the nice property of aligning incentives -- committers are naturally balanced across those who get implementations, not ppt, in shape for Colin/Dave acceptance as quickly possible. We see this as healthy. It has the nice property of reducing the "tyranny of the committer majority" risk -- no community process or design is perfect, but we're optimistic this approach will yield a positive outcome. We're in support of the D-E 'fresh repo' suggestion for kick-starting the D-E plan, we're quite motivated to make the controller project a balanced committer repo and we're quite motivated to see this move forward. If there is community consensus on this approach, great! If not, we look to you (Dave) to figure out a decision protocol at the next TSC call. -Kyle On Tue, May 14, 2013 at 7:37 AM, David Meyer <dmm@...> wrote: BTW, it has been pointed out that I mis-attributed the IOCTL idea to --------------------------------------------------------- R. Kyle Forster +1 (415) 990-2670 (m) kyle.forster@... |
|
|
|
David Meyer <dmm@...>
Kyle/all,
On Wed, May 15, 2013 at 7:04 AM, Kyle Forster <kyle.forster@...> wrote: Dave (and other members of the community) -In further considering the below, please find my comments in line. Please provide evidence to support these very serious accusations. IMO to begin with such an assumption/accusation is very damaging to the trust relationships that are core to a well functioning open source project. In fact these assertions/accusations create exactly the environment you claim to want to prevent. Again, please provide evidence of any "tyranny". These are serious accusations that require supporting evidence. The assertion that "issue is very real in other well known SDN open source projects" is simply not sufficient. We're in support of the D-E 'fresh repo' suggestion for kick-starting theAfter thinking about this for a few hours I went back and re-read the TSC charter for guidance. The following points are relevant to this thread: (i). Governance Section 5 of the TSC charter (http://www.opendaylight.org/project/tsc/charter) outlines the TSC's responsibility. In particular, it is not scope for the TSC to dictate the creation of any project. Specifically, "the TSC is responsible for simultaneous release dates, release quality standards, technical best practices (including the Development Process), monitoring technical progress, mediating technical conflicts between Committers and project leads, and organizing inter-project collaboration. The TSC will define ODP’s release vehicles." In summary, it is out of scope for the TSC to dictate the creation of any ODP project. (ii). Committers Section 7 of the TSC charter covers committers. Germain here is that "Initial projects that form ODP’s base may designate Committers.". In addition, "Committers select and vote for new Committers, subject to TSC approval.". What is clear from this section is that the TSC does not have the authority under our governance model to appoint committers to any project. Note that any appointment of committers by the TSC also violates the "meritocracy principle" that is core to any open source project. Summary: As I said in section (ii). below (New Project/Repository) and reassert here, "this is largely a political issue and as such is outside of the scope of the TSC". And while I initially thought a vote on this topic might resolve the problem upon further reflection and study I see that, as I originally stated, the creation of a New Project is out of scope for the TSC. Further, the TSC has no jurisdiction to appoint committers and in any event such an action would violate core open source principles. Finally, the ODP community is of course free to start any new project which will be subject to the ODP lifecycle. However, the creation of a new project and/or the appointment of committers is out of scope for the TSC (and in any event would violate core open source values). --dmm
|
|
|
|
Kyle Forster
David / all -
toggle quoted message
Show quoted text
I'd be happy to provide some stats on what a "tyranny of the majority committers" looks like from outside -- the asymmetric effort required to add interfaces to one project versus the other -- but let me do that on a separate thread in order to keep this one moving forward. I'm still optimistic that we can spot these patterns that we've seen in other SDN projects and protect against them in Daylight projects. In terms of next steps, if I read the charter notes you posted correctly, they would imply that committers and new project proposals are not TSC decisions but rather project-level decisions. Sounds healthy. I'd suggest then (to the community writ large) that David and Colin propose a new 'merged controller' project with the two of them as the leads and we follow the design and process route they've advocated on list for that project. For developers who need to build on the prod Floodlight components while we're porting them to the SAL, there is a repo for that. We're in that boat with the D-E design already. For developers who need to build on the SAL while it is in flight to get ready for the prod Floodlight components, there is a repo for that. We're in that boat with the D-E design already. For developers who need to build on a controller that starts empty but most resembles the end design state over the next few months, the new D-E project repo would fit that purpose. Their pushing will create natural demand pull for prioritizing the porting of SAL and Floodlight implementations in to the D-E end state. I'd argue this puts the power where it should be -- in the hands of developers writing apps along the D-E design -- and gives all of us in the controller-oriented projects a *lot* of incentive to serve them well. (It also gives David and Colin much deserved independent seats on the TSC. David was the author of Beacon, the codebase that spawned both implementations. It feels like the 'right thing' to do that he have a formal role there as well as in the community at large from his sweat if not his $$$.) -K On May 15, 2013, at 8:33 AM, David Meyer <dmm@...> wrote:
Kyle/all, |
|
|
|
Chris Wright <chrisw@...>
* Kyle Forster (kyle.forster@...) wrote:
In terms of next steps, if I read the charter notes you postedIndeed, that's the idea! ;) <snip "we've got a repo for that"> (It also gives David and Colin much deserved independent seats on theJust a formality, but it actually doesn't. This would require: - project proposal - creation to incubation - committers elect a project lead - promotion to core - the project lead is now on TSC On the other hand, we will have a few community representative seats on the TSC as well. So it's not entirely unreasonable to see both David on Colin sit on the TSC (recall, that a TSC seat isn't all that magical ;) David was the author of Beacon, the codebase that spawned bothDavid and Colin are fantastic assets to this community, no doubt! thanks, -chris |
|
|
|
Colin Dixon <ckd@...>
I just want to publicly say that I am not in favor of the creation of a third repository to house the merged code. For those of you who have been reading this list, that won't be news, but I wanted to make it as unambiguous as possible. |
|
|