Re: Opendaylight difficulties for a downstream project to integrate the release


Guillaume Lambert
 

Hi all

 

Ariel, thanks for the proposal. Unfortunately, I am off for a few days for family reasons.

If needed, I will let ATT be my proxy on that discussion today at the TSC.

Today the blocking point for us is the difficulty to run our functests with the kernel unstabilities, especially with netconf mocks (testtool 1.3.1 and honeynode).

Backward compatibility is not ensured today from my point of view.

You can refer to issues raised to the netconf ML yesterday and particularly https://jira.opendaylight.org/browse/NETCONF-552

 

We made adaptations to support RFC8345 https://git.opendaylight.org/gerrit/#/c/74593/

But until those issues are completely fixed, we cannot fully test them on Fluorine

Also, this raised a dilemma: should we ship or not unofficial openroadm models in an official Opendaylight release ?

 

Hope this helps

BR

Guillaume

 

 

 

From: Ariel Adam [mailto:aadam@...]
Sent: jeudi 2 août 2018 09:36
To: Abhijit Kumbhare; bf1936@...; LAMBERT Guillaume IMT/OLN
Cc: Juraj.Veverka@...; Daniel Farrell; CHAWKI Jamil IMT/OLN; ajoshipura@...; Philip Robb; Casey Cain; jzannos@...; mlemay@...; michel.biurrun@...; mb4962@...; Michael Vorburger; Stephen Kitt; IBRAHIM Houmed IMT/OLN; RENAIS Olivier IMT/OLN; Lisa Caywood; Luis Gomez; Robert Varga; Anil Vishnoi; Jamo Luhrsen; <tsc@...>; Tom Pantelis; Thomas Nadeau
Subject: Re: Opendaylight difficulties for a downstream project to integrate the release

 

Brian, putting aside the discussion on processes we agree that we should have done a better job communicating to everyone changes in the YANG infra especially since this was done late in the release. This is something we need to improve and we will.

 

Still, given that we are one week before the Fluorine feature freeze the immediate focus should be on solving the problem Guillaume and the team are facing.

Guillaume, appreciate if you can provide an updated status on the tasks you still need to complete and the problems which are blocking you.

 

Let's also allocate 10 minutes in today's TSC call to make sure you are getting the attention you need.

 

Thanks.  

 

 

 

On Thu, Aug 2, 2018 at 1:09 AM Abhijit Kumbhare <abhijit.kumbhare@...> wrote:

Hi Brian,

 

Thank you for your inputs. Two things to consider:

  • With the managed release process the self managed projects can release on their own schedule.
  • However I understand that TransportPCE and possibly some other self managed projects may want to be on the same release train or some similar solution. Some similar solution may be a train which arrives at a reasonable fixed time offset (say 3-4 weeks after the managed projects release).

 

About the second point – a key question is how to have the self managed projects or downstream consumers conduct their releases (self or simultaneous) when upstream projects make API changes. For managed projects this is not an issue as the release engineering team is doing the duty of herding the flock and making sure the upstream API changes are complied to. But this is an issue for self managed projects as well as downstream consumers. An obvious possible solution is to have a migration guide – but we still have to work through the details.

 

We have scheduled a TWS session coming Monday to discuss this and any other such issues for self managed projects are likely to have. It will be great if you can join.

 

BTW we did discuss this particular problem raised by TransportPCE briefly in the TSC meeting but in the end felt a larger discussion was needed – hence decided to go for it in a longer TWS session.

 

Thanks,

Abhijit

 

From: "FREEMAN, BRIAN D" <bf1936@...>
Date: Wednesday, August 1, 2018 at 1:07 PM
To: "guillaume.lambert@..." <guillaume.lambert@...>, Juraj Veverka <Juraj.Veverka@...>, Abhijit Kumbhare <abhijit.kumbhare@...>, Daniel Farrell <dfarrell@...>, CHAWKI Jamil IMT/OLN <jamil.chawki@...>
Cc: "Arpit Joshipura (ajoshipura@...)" <ajoshipura@...>, "Philip Robb (probb@...)" <probb@...>, "Casey Cain (ccain@...)" <ccain@...>, John Zannos <jzannos@...>, Mathieu Lemay <mlemay@...>, Michel Biurrun <michel.biurrun@...>, "BIRK, MARTIN" <mb4962@...>, "vorburger@..." <vorburger@...>, "skitt@..." <skitt@...>, IBRAHIM Houmed IMT/OLN <houmed.ibrahim@...>, RENAIS Olivier IMT/OLN <olivier.renais@...>, Lisa Caywood <lcaywood@...>
Subject: RE: Opendaylight difficulties for a downstream project to integrate the release

 

Sorry to interject late in the discussion but it just came up for me on a review of transportPCE.

 

It seems like the time between stable API’s and strongly dependent project (transportPCE is stongly dependent on core netconf and thus md-sal) in the timeline is rough on downstream projects.

 

TransportPCE is trying to do the right thing and be compatible with Flourine.

 

While rfc8345 is a good change,  it looks like the change for the ietf-networking  was introduced 16 weeks into the cycle  (was TSC-124 July 2 really when it was proposed ?)

 

Don’t want to start a huge discussion but it does seem like downstream projects need some leeway (or dont make these types of changes late in the cycle)

 

Brian

 

 

From: guillaume.lambert@... <guillaume.lambert@...>
Sent: Friday, July 13, 2018 5:15 AM
To: Juraj Veverka <Juraj.Veverka@...>; Abhijit Kumbhare <abhijit.kumbhare@...>; Daniel Farrell <dfarrell@...>; CHAWKI Jamil IMT/OLN <jamil.chawki@...>
Cc: Arpit Joshipura (ajoshipura@...) <ajoshipura@...>; Philip Robb (probb@...) <probb@...>; Casey Cain (ccain@...) <ccain@...>; John Zannos <jzannos@...>; Mathieu Lemay <mlemay@...>; Michel Biurrun <michel.biurrun@...>; FREEMAN, BRIAN D <bf1936@...>; BIRK, MARTIN <mb4962@...>; vorburger@...; skitt@...; IBRAHIM Houmed IMT/OLN <houmed.ibrahim@...>; RENAIS Olivier IMT/OLN <olivier.renais@...>; Lisa Caywood <lcaywood@...>
Subject: RE: Opendaylight difficulties for a downstream project to integrate the release

 

Hi Juraj

 

Thanks for you feedback.

The reason why we used the master branch is that we want to participate the Fluorine release.

We did not create a stable/oxygen Gerrit branch but we already tried what you proposed in the master branch by bumping our dependencies to Oxygen-SR2 (this is allowed for self-managed project).

This is in draft mode so I added you as a reviewer so you can take a look at it.

Once that done, the build works but we have a karaf runtime problem.

 

BR

Guillaume

 

 

From: Juraj Veverka [mailto:Juraj.Veverka@...]
Sent: vendredi 13 juillet 2018 10:42
To: LAMBERT Guillaume IMT/OLN; Abhijit Kumbhare; Daniel Farrell; CHAWKI Jamil IMT/OLN
Cc: Arpit Joshipura (ajoshipura@...); Philip Robb (probb@...); Casey Cain (ccain@...); John Zannos; Mathieu Lemay; Michel Biurrun; FREEMAN, BRIAN D <bf1936@...> (bf1936@...); mb4962@...; vorburger@...; skitt@...; IBRAHIM Houmed IMT/OLN; RENAIS Olivier IMT/OLN; Lisa Caywood
Subject: Re: Opendaylight difficulties for a downstream project to integrate the release

 

Hi Guillaume

 

I wonder if there is some particular reason why you need to use master branch on TransportPCE project ?

I would suggest that you create stable/oxygen branch in your project and you change all dependencies in your pom files so TransportPCE project will depend on oxygen SR2 artefacts.

 

When oxygen SR3 will be released, you should/may bump your project to oxygen SR3. 

 

Do not use master, because this branch is under development and API changes are expected.

 

Kind Regards

Juraj


From: guillaume.lambert@... <guillaume.lambert@...>
Sent: Wednesday, July 11, 2018 3:55:56 PM
To: Abhijit Kumbhare; Daniel Farrell; CHAWKI Jamil IMT/OLN
Cc: Arpit Joshipura (ajoshipura@...); Philip Robb (probb@...); Casey Cain (ccain@...); John Zannos; Mathieu Lemay; Michel Biurrun; FREEMAN, BRIAN D <bf1936@...> (bf1936@...); mb4962@...; vorburger@...; skitt@...; Juraj Veverka; IBRAHIM Houmed IMT/OLN; RENAIS Olivier IMT/OLN; Lisa Caywood
Subject: RE: Opendaylight difficulties for a downstream project to integrate the release

 

Hi all

 

Abhijit, I will be happy to join the TSC meeting tomorrow to talk about it.


Daniel, thanks a lot for your feedback. You’re probably right when you say it is just a matter of linking us to the right mailing-list thread.

I also trust you if you say the master branch is healthy and that the schedule is similar to the older release.
But our experience is that the current release process is probably not adapted for downstream projects.

 

About the healthy branch, how downstream projects can deal with the migration and do we know the master branch is healthy

, otherwise than following all core projects ML to solve the issues ?

This is clearly not sustainable when you have several strong dependencies such as the Netconf southbound connector.

As presented by Stephen and Michael during the last DDF, we ran last year into several issues with the SNAPSHOT branch core projects.

At that time, it was impossible for us to keep the branch up-to-date with the master SNAPSHOT since it was simply broken for days or weeks.

Thus, we had no other choice than to work on the previous stable release.
Now our code is finalized, we have difficulties to make it work with the new master branch, probably because we are facing several issues together.

We expected at least some examples or documentation to help us,  and also to confirm our POM configuration was OK but we didn’t find anything satisfying.

In my sense, the efforts should be better put on the project itself than sticking to its unstable dependencies.
That’s why we think downstream projects should have a different deadline after the kernel is stabilized.

 

About the release documentation, I know you put a lot of efforts to keep it maintained and that is really a good thing. I also looked at the URL you mentioned.
However, I noticed that this release management process documentation and its terminology kept on evolving after the start of the release.

And this is really a bit confusing.

It was said at the DDF we had to be ready before RC0. If I understand well what happened, the term RC0 has now been replaced by Code Freeze in Fluorine.

However, in the Oxygen documentation today, it is still not clear if RC0 is the final checkpoints or is 2 weeks before.

https://docs.opendaylight.org/en/stable-oxygen/release-process/managed-release.html

I quote:

“There will be an initial checkpoint two weeks after the start of the release, a midway checkpoints one month before RC0 and a final checkpoint at RC0.”

“At 2 weeks after RC0 a final checkpoint will be collected by the release team and presented to the TSC at the next TSC meeting.”

Also, in the wiki https://wiki.opendaylight.org/view/Simultaneous_Release:Release_Schedule_Framework

Code Freeze is still referenced as M4 and RC0 is given 2 weeks after.

 

Hope this helps

Guillaume

 

From: Abhijit Kumbhare [mailto:abhijit.kumbhare@...]
Sent: mardi 10 juillet 2018 20:41
To: Daniel Farrell; CHAWKI Jamil IMT/OLN
Cc: Arpit Joshipura (ajoshipura@...); Philip Robb (probb@...); Casey Cain (ccain@...); LAMBERT Guillaume IMT/OLN; John Zannos; Mathieu Lemay; Michel Biurrun; FREEMAN, BRIAN D <bf1936@...> (bf1936@...); mb4962@...; vorburger@...; skitt@...; Juraj.Veverka@...; IBRAHIM Houmed IMT/OLN; RENAIS Olivier IMT/OLN; Lisa Caywood
Subject: Re: Opendaylight difficulties for a downstream project to integrate the release

 

Jamil,

 

Let me know if you want it to be added to the this week’s TSC meeting agenda.

 

Thanks,

Abhijit

 

From: Daniel Farrell <dfarrell@...>
Date: Tuesday, July 10, 2018 at 9:50 AM
To: "jamil.chawki@..." <jamil.chawki@...>
Cc: "Arpit Joshipura (ajoshipura@...)" <ajoshipura@...>, "Philip Robb (probb@...)" <probb@...>, Abhijit Kumbhare <abhijit.kumbhare@...>, "Casey Cain (ccain@...)" <ccain@...>, LAMBERT Guillaume IMT/OLN <guillaume.lambert@...>, John Zannos <jzannos@...>, Mathieu Lemay <mlemay@...>, Michel Biurrun <michel.biurrun@...>, "FREEMAN, BRIAN D <bf1936@...> (bf1936@...)" <bf1936@...>, "mb4962@..." <mb4962@...>, "vorburger@..." <vorburger@...>, "skitt@..." <skitt@...>, "Juraj.Veverka@..." <Juraj.Veverka@...>, IBRAHIM Houmed IMT/OLN <houmed.ibrahim@...>, RENAIS Olivier IMT/OLN <olivier.renais@...>, Lisa Caywood <lcaywood@...>
Subject: Re: Opendaylight difficulties for a downstream project to integrate the release

 

Hello Jamil,

 

It sounds like you're having problems keeping your Self-Managed Project up-to-date with ODL's master branch. The change in your experience is related to our new Managed Release Process, which puts more responsibility on projects to keep things maintained. Previously, tight coupling between very active and less active projects forced the RelEng team and other active community members to do things like version bumps for projects they didn't actually have an interest in, just to keep autorelease building for everyone else. The new Managed Release process allows the RelEng team to focus on a smaller set of core OpenDaylight projects.

 

 

I replied to some details inline, but I think you'd be better off to take specific issues to the upstream mailing lists. In many cases I think it's just a matter of linking you to the relevant thread in the mailing list archives or an example of a similar patch.

 

Daniel

 

On Tue, Jul 10, 2018 at 12:17 PM <jamil.chawki@...> wrote:

Hello Arpit and Phil

I would like to share with you a real problem that we are facing with Opendaylight release process development and stability. As you know we are leading, with AT&T, the development of a Transport PCE project with an objective to manage Transport Networks (WDM/OpenROADM devices).

Our objective will be to publish finalized code in the next Fluorine Release but we are facing 2 problems:

·         The stability of master branch mainly for core component such as Netconf and MDSAL, we cannot test our code in a non-stable/working kernel core

Master branch looks healthy:

 

·         Short time period to deliver the code (one week before the end of Code Freeze Milestone).

There's 20 weeks from the start of the release cycle (branch cutting) to code freeze. I think that's similar to past releases.

 

We recommend to have different deadline for downstream project and  to freeze kernel projects (and publish downstream project configuration samples) 3 weeks before the code freeze  Milestone.

Guillaume will share our position during the next TSC meeting.

 

Looking forward to discussing it more on the TSC. Have you contacted the TSC mailing list to add it to the agenda?

 

 

 

Regards

Jamil  

 

 

De : LAMBERT Guillaume IMT/OLN
Envoyé : mardi 10 juillet 2018 17:39
À : CHAWKI Jamil IMT/OLN
Cc : RENAIS Olivier IMT/OLN
Objet : difficulties for a downstream project to integrate the release

 




The release planning has a lot evolved between Oxygen and Fluorine and we noticed recent evolution in the deadlines.
Self-managed / unmanaged projects that wants to participate the release must now be ready one week before the Code Freeze, so for the 31st July.
The transportPCE project is in that case but we are facing unexpected difficulties with our dependencies.

When we use the nitrogen dependencies version in our pom, the project builds correctly and is fully functional or almost
However, when we want to make it work on the current master branch, a lot of dependencies version problems appears.
It seems that we are not the only one project in that case since some Oxygen and Fluorine projects have their autorelease jobs still failing today.
We managed to make the compilation process work but we are currently facing a karaf runtime problem with the netconf connector, even with the latest stable dependencies published.

We are still looking for a solution to this issue but the job to get a working pom configuration is quite complex since we have no documentation nor working example that use the current southbound netconf connector.
The nexus nomenclature has tool evolved. Referenced jar packages name do not mention anymore the corresponding release name.
Some versions have even absent from the Nexus UI search engine. Thus,  there is no other way than to browse manually the repo tree to solve dependencies problems.
But combinations possibilities are too big to be all tested.
There is also a second issue that is surprising us.
Latest odlparent dependencies are using 3.1.X versions that can be declared in karaf4 features folder binding-parent dep, but the usage  is to use a 2.0.X version in the pom root project.

We are now less than 3 weeks before the deadline and we don't think we will be able to solve this issue without external help.
For future release planning to come, we may consider to freeze kernel projects and publish downstream project configuration samples before this deadline to avoid this kind of problem.

Thanks in advance for your help

BR
Guillaume

_________________________________________________________________________________________________________________________
 
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
 
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________
 
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
 
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________
 
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
 
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

Join TSC@lists.opendaylight.org to automatically receive all group messages.