Hi TSC ex-colleagues,

As I promised, here is a list of topics for ODL modernization. This is all that came to my mind now but it could be more, the goal here is really to trigger some brainstorm and discussion.

1) Use cases:

Our YANG based platform made it easy for ODL to become the SDN controller of choice for HW devices supporting NETCONF and other control protocols like BGP-LS, PCEP, OPENFLOW where multi-vendor is important. However it made it hard to have a role in the cloud where YANG and protocols like NETCONF, BGP-LS, PCEP are almost not existent and multi-vendor is not that important. Now, given the amount of money and effort that is currently put in the cloud, it would make sense for ODL to at least participate in some cloud use case. For example, I believe ODL can still play some role in the hybrid cloud use case where HW devices need to talk to cloud devices, ODL could take care of the HW devices while another open source controller could take care of the cloud devices.

2) SW Platform:

The OSGI/Karaf platform was state-of-the-art 10 years back where there was no real micro-services platforms like K8s, but now it is just obsolete. In the new paradigm of micro-services, applications are loose-coupled and they are mostly self-contained (e.g. run their own processes within a container) although they can share some common resources like a database, a message broker, an API gateway, etc.

For ODL to fit in the new paradigm, we would need to:

- Replace ODL distribution with ODL applications that can run in their own container: NETCONF, BGP, PCEP, TPCE, etc
- Consolidate kernel repos and jars: Existing ODL applications share a common code called kernel. Today we have ~6 repositories and a bunch of jars for the kernel where there is a single person/organization maintaining the code.
- Replace Karaf/OSGI framework with something more actual to plumb the java code and jars together (e.g. spring).

3) Infrastructure:

Here is kind of obsolete too. Some ideas to renew the build and test infrastructure:

Build pipeline:
- Move from JJB (not maintained anymore) to Jenkins pipelines or similar (work is ongoing).
- Move to a continuous release process where a merge in master produces a new release in the artifact repository (staging). This simplifies a lot the release process: just move artifacts from staging to release repository.
- Every ODL application (NETCONF, BGP, PCEP, TPCE, etc) should generate a container automatically after a merge in master. We should be testing this vs ODL distribution.

System test:
- Robot was the best open source system test framework 10 years back where organizations had developers and system engineers separated. Things have changed a lot since and many agile organizations nowadays have developers doing system integration and system test code apart from writing product features. Robot framework is good for system integrators with basic or non coding skills but bad for developers that have to ramp up in a new language that soon find very limiting. This is why I think at this moment it would be good to switch to something like pytest for example.
- Leverage K8s to do multi-application and scale testing.

BR/Luis

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

On 06/04/2022 01:21, Daniel de la Rosa wrote:

Hello Robert and all
it looks like we still have several issues
https://jenkins.opendaylight.org/releng/view/autorelease/job/integration-distribution-test-sulfur/190/ <https://jenkins.opendaylight.org/releng/view/autorelease/job/integration-distribution-test-sulfur/190/>
are we still tracking some of these issues in jira tickets?

Yes, I think most of the issues are already fixed, but I am waiting for https://jira.linuxfoundation.org/plugins/servlet/theme/portal/2/IT-23826 to issue the corresponding releases.

With the upgrade guide complete, I think we are good to branch, though.

Regards,
Robert

On 31/03/2022 17:41, Robert Varga wrote:

Alright, everything has been merged up.
int/dist and autorelease need to have their jobs updated to use maven-3.8, please review the patches here:
https://git.opendaylight.org/gerrit/c/releng/builder/+/100304
https://git.opendaylight.org/gerrit/c/releng/builder/+/100306
I am writing up the release notes and upgrade guide next.

So looking at https://jenkins.opendaylight.org/releng/job/integration-distribution-test-sulfur/185/ we have plenty of badness.

The core problem is that a ton of test suites are depending on OPERATIONAL_API, which points to old RESTCONF, which is no longer installed by default. I have https://git.opendaylight.org/gerrit/c/releng/builder/+/100359 to install it for now, but we need to eventually (very soon) ditch it.

Luis, any idea on how to fix that in a sensible manner?

Also, RF is now reporting a ton of ugly 'Keyword 'BuiltIn.Run Keyword Unless' is deprecated.' messages -- I'll take a peek next week unless somebody beats me to it.

Finally, I am tracking two regressions in MD-SAL (already fixed) and one in NETCONF (next on the list), but I'd like to get a better reading on CSIT before rolling out the corresponding releases out.

Regards,
Robert

On 31/03/2022 01:20, Robert Varga wrote:

On 29/03/2022 01:13, Robert Varga wrote:

On 28/03/2022 00:10, Robert Varga wrote:

On 24/03/2022 01:24, Daniel de la Rosa wrote:

On Wed, Mar 23, 2022 at 3:47 PM Robert Varga <nite@... <mailto:nite@...>> wrote:
That’s impressive. Ok thanks and it is all good. We will have a very solid sulfur release

A quick update here. We are on the home stretch now, with 31 callsites still being bad. About a third are duplicate (e.g. they are the same weirdness across bierman20/rfc8040). They will need some amount of TLC, which I am not in position to give right now, sorry :(

Okay, so I am down to 3 call sites, which fall into two categories:

1. old RESTCONF's URI parser leafref support. I think I can perform sufficient surgery to make this work without having to rewrite that thing (as was done for new RESTCONF, where this is a breeze)

2. two instances of a weird interaction with XML codec. It is not that weird, I understand what is going on, it just needs a bit of more work.

I will need another day to deal with them and, fingers crossed, UTs should agree.

This is now cleared, netconf-3.0.0 is out there. Most of the MSI update patches are done -- except openflowplugin.
Upgraded spotbugs is finding some badness which was introduced in https://git.opendaylight.org/gerrit/c/openflowplugin/+/91313 and I need to figure out how to fix it.
BGPCEP still has a four contributions left for review, I will take care of that tomorrow.

Alright, everything has been merged up.

int/dist and autorelease need to have their jobs updated to use maven-3.8, please review the patches here:

https://git.opendaylight.org/gerrit/c/releng/builder/+/100304
https://git.opendaylight.org/gerrit/c/releng/builder/+/100306

I am writing up the release notes and upgrade guide next.

Bye,
Robert

On 29/03/2022 01:13, Robert Varga wrote:

On 28/03/2022 00:10, Robert Varga wrote:

On 24/03/2022 01:24, Daniel de la Rosa wrote:

On Wed, Mar 23, 2022 at 3:47 PM Robert Varga <nite@... <mailto:nite@...>> wrote:
That’s impressive. Ok thanks and it is all good. We will have a very solid sulfur release

A quick update here. We are on the home stretch now, with 31 callsites still being bad. About a third are duplicate (e.g. they are the same weirdness across bierman20/rfc8040). They will need some amount of TLC, which I am not in position to give right now, sorry :(

Okay, so I am down to 3 call sites, which fall into two categories:
1. old RESTCONF's URI parser leafref support. I think I can perform sufficient surgery to make this work without having to rewrite that thing (as was done for new RESTCONF, where this is a breeze)
2. two instances of a weird interaction with XML codec. It is not that weird, I understand what is going on, it just needs a bit of more work.
I will need another day to deal with them and, fingers crossed, UTs should agree.

This is now cleared, netconf-3.0.0 is out there. Most of the MSI update patches are done -- except openflowplugin.

Upgraded spotbugs is finding some badness which was introduced in https://git.opendaylight.org/gerrit/c/openflowplugin/+/91313 and I need to figure out how to fix it.

BGPCEP still has a four contributions left for review, I will take care of that tomorrow.

Regards,
Robert

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

On 28/03/2022 00:10, Robert Varga wrote:

On 24/03/2022 01:24, Daniel de la Rosa wrote:

On Wed, Mar 23, 2022 at 3:47 PM Robert Varga <nite@... <mailto:nite@...>> wrote:
That’s impressive. Ok thanks and it is all good. We will have a very solid sulfur release

A quick update here. We are on the home stretch now, with 31 callsites still being bad. About a third are duplicate (e.g. they are the same weirdness across bierman20/rfc8040). They will need some amount of TLC, which I am not in position to give right now, sorry :(

Okay, so I am down to 3 call sites, which fall into two categories:

1. old RESTCONF's URI parser leafref support. I think I can perform sufficient surgery to make this work without having to rewrite that thing (as was done for new RESTCONF, where this is a breeze)

2. two instances of a weird interaction with XML codec. It is not that weird, I understand what is going on, it just needs a bit of more work.

I will need another day to deal with them and, fingers crossed, UTs should agree.

Regards,
Robert

On 24/03/2022 01:24, Daniel de la Rosa wrote:

On Wed, Mar 23, 2022 at 3:47 PM Robert Varga <nite@... <mailto:nite@...>> wrote:
That’s impressive. Ok thanks and it is all good. We will have a very solid sulfur release

A quick update here. We are on the home stretch now, with 31 callsites still being bad. About a third are duplicate (e.g. they are the same weirdness across bierman20/rfc8040). They will need some amount of TLC, which I am not in position to give right now, sorry :(

I expect to have them cleared tomorrow, Tuesday the latest, at which point I'll activate the supercommitter rights and roll things out.

The upgrade guide will come right after that, e.g. Tue or Wed.

Regards,
Robert

Hello Robert, all,

On 21/03/2022 14:45, Olivier Dugeon via lists.opendaylight.org wrote:

bgpcep is ready for integration, but netconf is not -- as it turns the vast majority of patches dealing with SchemaNode.getPath() caller removal had to be reverted because ... let's say they were naive.

Not yet for BGPCEP. I discover some late bugs on new PCE server last week. Patch is ready. I'll submit it this afternoon, just right after last verification.

Latest patches have been submitted:

• Correct bugs in Path Manager: https://git.opendaylight.org/gerrit/c/bgpcep/+/100205/4
• Add Close Loop mechanism to PCE Server: https://git.opendaylight.org/gerrit/c/bgpcep/+/100206/5
• Add Exclude and Include Route for Path Computation: https://git.opendaylight.org/gerrit/c/bgpcep/+/100247/1

The first two have been reviewed and are ready to be merged. Latest one needs to be reviewed.

The only point that is remaining for BGPCEP, is to clean the compilation from various warning message. I don't know if we include these now or wait for Sulfur SR1.

Regards

Olivier

Alright, anyway it's going to take a couple of days to sort out the RESTCONF stuff, so that (and the javadoc jobs) are the real blockers.

Regards,
Robert

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

--