1. Project-Proposals
  2. Topics

[controller-dev] [yangtools-dev] Proposal to spin out MD-SAL

Edward Warnicke
 

Colin,
     Comments inline...

On Thu, Apr 2, 2015 at 1:10 PM, Colin Dixon <colin@...> wrote:
So, I was thinking about this again recently and I realized the discussion largely petered out.

I have to say, honestly, I don't really feel like we've gotten satisfactory answers to the key questions that people asked:

* Will this cause issues with needed to merge pairs of patches that are inter-dependent? The claim is no, but frankly I don't believe it short of claiming we won't be changing the APIs.

It should actually reduce the cross patch issue as things get moved into more logical groupings.
 

Actually, it might be better to let past code changes guide where project (or sub-project) boundaries should be. That is, produce a graph where each file is a node and there's add an edge between each pair of nodes when they're both affected by a commit. Then repeatedly find min-cuts in this graph until you get the number of sub-projects you think you want.

* What impact will this have with making it possible for new developers to get involved?

It should make it a fair bit easier, as you will have distinct logical components, together, and without a bunch of unrelated stuff to wade through.
 

* Is there a compelling reason that this couldn't be done by simply using a bit of discipline and good directory structure? That is, do we really need the extra mechanism of new repositories or do we just need logical sub-projects?

Yes:

1)  Build times - local, merge, verify
2)  Channels of patches in gerrit - you can easily see what goes what project... better granularity there
3)  Better clustering of committer expertise
4)  Well defined scope
5)  Simpler path to committerness (a logical outgrowth of better defined scope)

We are seeing some of the benefits of these sorts already in neutron, which was using the 'subdir' method you suggested,
and got a shot in the arm along these axis when it spun out.

Basically we tried 'subdir' with neutron, and then we tried 'spin out a project'.  It turns out that 'spin out the project' is *much* better for these reasons and probably
several others I'm forgetting.  (oh, it also seems to encourage more participation as well... a lovely side benefit :) ).

Ed
 


--Colin


On Thu, Feb 19, 2015 at 12:09 PM, Robert Varga <nite@...> wrote:
I completely agree about the pain. The difference now is that we are not talking about models, but rather Java APIs and for both YT and MD-SAL APIs we think the APIs are stable enough to be able to evolve compatibly.

Bye,
Robert


On 02/19/2015 04:28 PM, Anil Vishnoi wrote:
I agree with Colin's comment. We face the similar issue with Openflowplugin project, where NSF/Models were living in controller and their implementation was in openflowplugin project. Everytime we pushed patch, we need to make sure that it get into controller first, it's artifact get published and then you push the openflowplugin project. 

Ed shared this pain with us :-), because we used to poke him a lot to merge patches to the controller, so that we can merge patch to openflowplugin. By sharing this experience i am not disagreeing to anything, just putting forward concerns that we should discuss before we go forward on this path.

On Thu, Feb 19, 2015 at 8:50 PM, Colin Dixon <colin@...> wrote:
I'm curious how much of this could be done by having logical sub-projects that are well-organized and well-separated within the YANG tools repository. I understand that we'd like to factor apart the various pieces of functionality, but it's unclear what the advantage of factoring them apart into different projects vs. merely logically separating them within a project is.

In particular, there are some clear disadvantages to breaking things up into multiple projects. The most obvious of them being that if the projects are somewhat tightly coupled (as I would imagine these are), it will require us to frequently push groups of patches to multiple projects when things change and cause (hopefully brief) periods of inconsistency as patches are applied in one project but not (yet) applied in the other projects.

--Colin


On Thu, Feb 19, 2015 at 3:23 AM, Robert Varga <nite@...> wrote:
Hello everyone,

The YANG tools project would like to spin out a project containing the Java bindings and the code generators related to them. After analyzing where they fit conceptually, we have found that the Java bindings at this stage are tightly bound to the Binding Aware MD-SAL and thus they should live together, so they can be co-evolved with more ease.

Since the overall tune of the controller is trimming down, rather than dropping a large chunk of code into it, we would like to create a project, which will combine:
- MD-SAL binding-independent APIs and SPIs
- MD-SAL binding-aware APIs and SPIs
- MD-SAL binding-aware-to-binding-indepent connector
- YANG tools' Java code generators and support classes
to create a well-defined abstraction layer, without tying it to any particular implementation.

The details of the proposal are captured here: https://wiki.opendaylight.org/view/Project_Proposals:MD-SAL, comments and contributions are more than welcome.

Thanks,
Robert
_______________________________________________
controller-dev mailing list
controller-dev@...
https://lists.opendaylight.org/mailman/listinfo/controller-dev


_______________________________________________
yangtools-dev mailing list
yangtools-dev@...
https://lists.opendaylight.org/mailman/listinfo/yangtools-dev




--
Thanks
Anil



_______________________________________________
controller-dev mailing list
controller-dev@...
https://lists.opendaylight.org/mailman/listinfo/controller-dev
Colin Dixon
 

So, I agree with all of your points at the end (1-5+).

However, I still feel like we haven't heard a compelling case as to why this is going to look more like Neutron (which was very successful) and less like OpenFlow plugin (which was repeatedly and annoyingly blocked by having to have cross-project patches until we also moved the NSFs to the project).

--Colin


On Thu, May 7, 2015 at 1:36 PM, Edward Warnicke <hagbard@...> wrote:
Colin,
     Comments inline...

On Thu, Apr 2, 2015 at 1:10 PM, Colin Dixon <colin@...> wrote:
So, I was thinking about this again recently and I realized the discussion largely petered out.

I have to say, honestly, I don't really feel like we've gotten satisfactory answers to the key questions that people asked:

* Will this cause issues with needed to merge pairs of patches that are inter-dependent? The claim is no, but frankly I don't believe it short of claiming we won't be changing the APIs.

It should actually reduce the cross patch issue as things get moved into more logical groupings.
 

Actually, it might be better to let past code changes guide where project (or sub-project) boundaries should be. That is, produce a graph where each file is a node and there's add an edge between each pair of nodes when they're both affected by a commit. Then repeatedly find min-cuts in this graph until you get the number of sub-projects you think you want.

* What impact will this have with making it possible for new developers to get involved?

It should make it a fair bit easier, as you will have distinct logical components, together, and without a bunch of unrelated stuff to wade through.
 

* Is there a compelling reason that this couldn't be done by simply using a bit of discipline and good directory structure? That is, do we really need the extra mechanism of new repositories or do we just need logical sub-projects?

Yes:

1)  Build times - local, merge, verify
2)  Channels of patches in gerrit - you can easily see what goes what project... better granularity there
3)  Better clustering of committer expertise
4)  Well defined scope
5)  Simpler path to committerness (a logical outgrowth of better defined scope)

We are seeing some of the benefits of these sorts already in neutron, which was using the 'subdir' method you suggested,
and got a shot in the arm along these axis when it spun out.

Basically we tried 'subdir' with neutron, and then we tried 'spin out a project'.  It turns out that 'spin out the project' is *much* better for these reasons and probably
several others I'm forgetting.  (oh, it also seems to encourage more participation as well... a lovely side benefit :) ).

Ed
 


--Colin


On Thu, Feb 19, 2015 at 12:09 PM, Robert Varga <nite@...> wrote:
I completely agree about the pain. The difference now is that we are not talking about models, but rather Java APIs and for both YT and MD-SAL APIs we think the APIs are stable enough to be able to evolve compatibly.

Bye,
Robert


On 02/19/2015 04:28 PM, Anil Vishnoi wrote:
I agree with Colin's comment. We face the similar issue with Openflowplugin project, where NSF/Models were living in controller and their implementation was in openflowplugin project. Everytime we pushed patch, we need to make sure that it get into controller first, it's artifact get published and then you push the openflowplugin project. 

Ed shared this pain with us :-), because we used to poke him a lot to merge patches to the controller, so that we can merge patch to openflowplugin. By sharing this experience i am not disagreeing to anything, just putting forward concerns that we should discuss before we go forward on this path.

On Thu, Feb 19, 2015 at 8:50 PM, Colin Dixon <colin@...> wrote:
I'm curious how much of this could be done by having logical sub-projects that are well-organized and well-separated within the YANG tools repository. I understand that we'd like to factor apart the various pieces of functionality, but it's unclear what the advantage of factoring them apart into different projects vs. merely logically separating them within a project is.

In particular, there are some clear disadvantages to breaking things up into multiple projects. The most obvious of them being that if the projects are somewhat tightly coupled (as I would imagine these are), it will require us to frequently push groups of patches to multiple projects when things change and cause (hopefully brief) periods of inconsistency as patches are applied in one project but not (yet) applied in the other projects.

--Colin


On Thu, Feb 19, 2015 at 3:23 AM, Robert Varga <nite@...> wrote:
Hello everyone,

The YANG tools project would like to spin out a project containing the Java bindings and the code generators related to them. After analyzing where they fit conceptually, we have found that the Java bindings at this stage are tightly bound to the Binding Aware MD-SAL and thus they should live together, so they can be co-evolved with more ease.

Since the overall tune of the controller is trimming down, rather than dropping a large chunk of code into it, we would like to create a project, which will combine:
- MD-SAL binding-independent APIs and SPIs
- MD-SAL binding-aware APIs and SPIs
- MD-SAL binding-aware-to-binding-indepent connector
- YANG tools' Java code generators and support classes
to create a well-defined abstraction layer, without tying it to any particular implementation.

The details of the proposal are captured here: https://wiki.opendaylight.org/view/Project_Proposals:MD-SAL, comments and contributions are more than welcome.

Thanks,
Robert
_______________________________________________
controller-dev mailing list
controller-dev@...
https://lists.opendaylight.org/mailman/listinfo/controller-dev


_______________________________________________
yangtools-dev mailing list
yangtools-dev@...
https://lists.opendaylight.org/mailman/listinfo/yangtools-dev




--
Thanks
Anil



_______________________________________________
controller-dev mailing list
controller-dev@...
https://lists.opendaylight.org/mailman/listinfo/controller-dev
Robert Varga
 

On 06/03/2015 09:48 PM, Colin Dixon wrote:
So, I agree with all of your points at the end (1-5+).

However, I still feel like we haven't heard a compelling case as to why this is going to look more like Neutron (which was very successful) and less like OpenFlow plugin (which was repeatedly and annoyingly blocked by having to have cross-project patches until we also moved the NSFs to the project).
Well, the fact that current cross-project synchronization between yangtools and controller are not visible is because we can coordinate and move quickly. That certainly does not mean they do not occur. Last time this was needed was ... https://git.opendaylight.org/gerrit/#/c/21220/ and https://git.opendaylight.org/gerrit/#/c/21223/.

As previously stated, binding-related classes in yangtools are much closer to MD-SAL classes in controller than they are to anything else in yangtools. Peering into what are candidate work items, I do not see this changing in at least one release. There is only one dependency (in flexible types), which is easily expressed as an inter-project dependency, with more than one consumer.

The move of models was the most important one, but the move of NSFs allowed them to be co-evolved -- there is no inventory manager in Lithium... I will also argue that a lot of the issues seen in OF-land run deeper than code location, but that's a different topic.

Bye,
Robert

--Colin

1 - 3 of 3