Re: my perspective on where we are with the single controller code base issue


Kyle Forster
 

Dave (and other members of the community) - 

The folks that work here at BSN are here to make Daylight as successful as we can, but i) we owe transparency about our over-arching concern (that we think is valid for a large swath of the community), ii) we'd like to go to a few facts and iii) we'd like to chart a productive path forward. 

-------

Transparency -- 

The over-arching concern that we (BSN folk) have tried to mitigate with design and process proposals is a "tyranny of the committer majority" problem in the controller project, a situation where a highly concentrated group of committers blocks well intentioned progress by teams around them whose applications call for different design trade-offs and timing priorities.

This issue is very real in other well known SDN open source projects where competing companies are regularly locked out from material commits.  The result is an awkward amalgamation of open and closed source components across a handful of companies now, a trail of open source that isn't broadly used in production and a lot of confusion.

The problem isn't because the concentrated group of committers in these projects have been evil, or dumb, or mean.  They simply prioritize designs for the needs they see and add friction for the needs they don't until those needs go away.  While subjective, I submit to the list that the same patterns are emerging in Daylight already, and it will take extraordinary effort by motivated individuals in the months ahead to avoid this path.

-------

Let's go to facts -- 

First, the D-E plan you linked to above does *not* specify a starting repo in writing, but rather states explicitly on the page that the authors disagree on this point, a point that has been re-emphasized in the public threads.  D-E plan, full steam ahead!  Which D-E plan starting repo?

Second, on community support, by my count of emails on list, I came up with 10 organizations voicing support for a fresh repo / committer list and 3 against.  Have we asked for on-list support from others who share our concerns privately?  Yes!  But even if we filter by TSC organizations (community+$$$), we came up with 3 in favor, 3 against, 3 no explicit vote.

--------

How to move forward?  --

We are in strong favor of the fresh repo and blank committer list with Dave and Colin as the first two committers.  Let them invite other committers to the "merged controller repo" along with their code as it flows to their design.  This is the suggestion I believe that both Dave and Colin have supported as their own compromise as the authors of the D-E plan.  

It has the nice property of parallelizing work -- as we've drilled down, there is a lot to do on SAL to make it ready for production-grade Floodlight components, and a lot to do on making production-grade Floodlight components SAL-ready.  We're signed up for a lot there.

It has the nice property of aligning incentives -- committers are naturally balanced across those who get implementations, not ppt, in shape for Colin/Dave acceptance as quickly possible.  We see this as healthy. 

It has the nice property of reducing the "tyranny of the committer majority" risk -- no community process or design is perfect, but we're optimistic this approach will yield a positive outcome.

We're in support of the D-E 'fresh repo' suggestion for kick-starting the D-E plan, we're quite motivated to make the controller project a balanced committer repo and we're quite motivated to see this move forward.  If there is community consensus on this approach, great!  If not, we look to you (Dave) to figure out a decision protocol at the next TSC call.

-Kyle



On Tue, May 14, 2013 at 7:37 AM, David Meyer <dmm@...> wrote:
BTW, it has been pointed out that I mis-attributed the IOCTL idea to
Chris. That was my mistake. That said, the point I was making still
holds, namely, that there exist some ideas (and there will be more) to
deal with the near term deficiencies, perceived or otherwise, with the
SAL.

--dmm


On Tue, May 14, 2013 at 9:09 AM, David Meyer <dmm@...> wrote:
> All,
>
> As you know today (05.14.2013) was the day on which I wanted to
> have the single controller code base issue resolved. What I had
> said is that if we couldn't get the projects to agree on a way
> forward that we as a TSC would vote on the issue. In the mean
> time we have reached general consensus that the Dixon-Erickson
> OpenDaylight Merged Controller Proposal (DE) (see
> https://wiki.opendaylight.org/view/Dixon-Erickson_OpenDaylight_Merged_Controller_Proposal).
> I want to be very clear here that that plan is distinct from what
> is being called Proposed Dixon-Erickson Execution Plan (see
> https://wiki.opendaylight.org/view/Proposed_Dixon-Erickson_Execution_Plan). So
> from this perspective the community has come together over this
> general direction. This functionally obviates the need for a
> direct vote.
>
> There have been quite a few concerns raised both publicly and to
> me directly in private email. I'd like to review those here and
> provide my perspective.
>
> (i). The SAL
>
> It is generally agreed that there is work left to be done on the
> SAL, and that there may be instances in which, (in the near term)
> we need a mechanism to provide a kind of "SAL bypass". Chris'
> IOCTL proposal is one such idea. What I want to point out here is
> the the ODP is evolving and as such we expect the architecture to
> not not improve technically but converge on the needed
> functionality/generality over time. This is a completely natural
> state of affairs for any software project. The summary is that we
> can provide short term mechanisms that bridge any perceived or
> real functionality gaps in the SAL (again, Chris' IOCTL idea
> might be one such mechanism).  So while the OSCP folks have
> argued pretty strongly against the SAL (both publicly and
> privately), the DE proposal endorses the SAL concept and the
> OpenDaylight Controller folks consider the SAL to be a
> fundamental architectural construct. That said, evolution of the
> SAL (be it the model driven approach, bypass, etc) is an issue
> for folks who are writing SAL applications (committing code) to
> resolve.
>
> (ii). New Project/Repository
>
> This issue is really about the fairness of commits to the
> controller code base. This was stated explicitly to me privately
> and publicly by Glenn Ricart and Guru Parulkar on the discuss
> list. On the other hand, a simple majority of TSC members (and
> many in the community) have expressed the opinion that they would
> prefer moving forward with the plan outlined in the DE-proposal,
> namely start with the OpenDaylight controller and evolve that as
> needed. I will note here that there is no objective evidence that
> there is a fairness issue with commits to any candiate code base
> (it would hard for that to be the case in any event as we haven't
> had the chance to really get going). Finally, since this is
> largely a political issue it is outside of the scope of the TSC
> and should be "policed" by the community.
>
> As an aside, I will also note here that several community
> members have expressed concern that the new project
> approach is quite expensive and has little benefit beyond
> the political/perception issue mentioned above.
>
>
> (iii)  Timing
>
> The timing issue, namely that there is (is not) a rush to get the
> ODP project up and running is one that has also been raised both
> publicly and privately. I have been told (again, both publicly
> and privately) by the OSCP (BSN) folks that they have time
> pressure to port their applications to the ODP controller. That
> is a principle rationale behind the Layered API proposal
> (https://wiki.opendaylight.org/view/Layered_API_Merged_Controller_Proposal).
> In addition, the same team is also arguing that  we are in no rush and
> in fact any date for resolving the controller issue is not only
> artificial but damaging to the community; this exact sentiment was
> contained in a private communication from Rob Sherwood to me dated
> Tue, May 14, 2013 at 1:38 AM. In the spirit of transparency I am copying the
> relevant text from that email here:
>
> "I know that in your heart you believe what you're doing is the
>  right thing for the community, and that's why I've held off
>  making a stink, but as chair, I'm not sure what you're doing is
>  in terms of pushing hard on this artificial deadline (that you
>  said originally was not set in stone but now it is) or being
>  opaque in terms of the actual process is really helping the
>  community."  -- Rob Sherwood to dmm on Tue, May 14, 2013 at 1:38
>  AM.
>
>  (this text of course also contains criticisms of me in my role
>  as chair to which I will return but I want to focus here on the
>  timing issue).
>
> The summary here is that it has been argued by the OSCP team that
> we accept their proposal
> (https://wiki.opendaylight.org/view/File:LayeredAPIProposal.pdf)
> because there is time pressure to port applications while at the
> same time it is being argued that any time frame is damaging,
> artificial, or worse. On the other hand, a simple majority of the
> TSC have stated publicly (on the discuss list) that resolving the
> controller code base issue of primary importance and they would
> like to move forward with an interpretation of DE that starts
> with the existing OpenDaylight controller code base.
>
> It is clear from these inconsistencies that the timing issue is
> largely a non-issue for the ODP community at large; in addition
> there is clear benefit to moving forward with the DE approach now. One
> might also note here that we've made very little progress over
> the last many weeks and it is time to start, now.
>
> (iv).  Finally, if anyone has problems with me personally and/or the
> way TSC is being run please air those publicly on the
> discuss list. I will not be responding to emails such as the one
> quoted above, lobbying by private phone call, or the
> like. Again, if you have complaints about the way the TSC
> is being run please discuss those on the discuss or tsc lists.
>
> Summary: It is clear that there is additional evolution that is
> needed for the SAL. No one is arguing otherwise. The new
> project/repository issue is really about concern that commits
> won't be fair (again, both Glenn and Guru raised this); however
> there is no evidence that this is or will be the case and in any
> event is not a technical issue and as such is out of scope for
> the TSC. Finally, the timing issue is largely a commercial issue
> (the need to port applications) and also has been argued on both
> sides by the same people and as such I consider those arguments
> largely moot; there is a real need to get the consortium going
> (hence the date) but that is a wholly different argument.
>
> --dmm
_______________________________________________
TSC mailing list
TSC@...
https://lists.opendaylight.org/mailman/listinfo/tsc



--
---------------------------------------------------------
R. Kyle Forster
+1 (415) 990-2670 (m)
kyle.forster@...

Join {TSC@lists.opendaylight.org to automatically receive all group messages.