Query regarding internship project - YangModelsPublish


Luis Gomez
 



On May 10, 2019, at 9:31 AM, Luis Gomez <ecelgp@...> wrote:



On May 9, 2019, at 12:44 PM, rohan julka <rohanjulka19@...> wrote:

Hi ,
Is it possible to contribute the Yang models using submodules ?https://github.com/YangModels/yang#contributions-via-submodules  , this way the sync script will be executed by ODL .

We can use submodule if there is some benefit in that, from what I read in the link you posted, it should be possible to use shell/commandline to create a PR through the direct contribution method. I guess we can decide best method by also talking with the YanModels community.


Now if the models needs to be pushed in the master folder every week , the sync script https://github.com/YangModels/yang/pull/557 will not work as it requires the auto release path which already contains the submodules of all the repos updated. So we can either submodule each ODL project and pass it to the sync script or create a script which will go through each ODL project repo , get the yang models and push all the models to yang models repo.

Yeah, I am not sure current AR submodules cover all projects we want in ODL, specially the MRI projects that are not part of AR. Best place to see project and version for a given branch is this file in distribution:

https://git.opendaylight.org/gerrit/gitweb?p=integration/distribution.git;a=blob;f=artifacts/upstream/properties/pom.xml;h=e4c41bbb29400d2e53166b8cf1e19d4c4baa4067;hb=HEAD


After that as you suggested the models list (in the form of google sheets?) can be reverted on the common dev mailing list for verification dev@... .

Yes, once we prepare a candidate list of models we can share that with the community through that list.


Regards,
Rohan



From: rohan julka
Sent: 08 May 2019 02:48
To: Luis Gomez
Subject: Re: [integration-dev] Query regarding internship project - YangModelsPublish

Thank you so much for providing the details and also the resources earlier, now I have a more clearer
picture of what I have to do. I will get back to you soon with my proposed solutions. 

Regards, 
Rohan 
On Wed, May 8, 2019, 2:38 AM Luis Gomez <ecelgp@...> wrote:
Basically complete #556 & #557 but I would probably rethink the full process here, like:

1) Make sure the sync script proposed in https://github.com/YangModels/yang/pull/557 collects the right models from ODL. For that we will need to revert to community with the list of models and see if everybody agrees with.
2) Try to run the sync script in ODL so we have control over the trigger. This may require some handle/permission from YangModels folks.
3) Think how often we want to push the models to YangModels: in theory it should be enough to push every time a major ODL release is produced but the YangModels CI can also be used to verify yang model themselves so we probably want something like pushing master models once a week in a folder called master and every major release (we branch from master every 6 months) push a folder (e.g. sodium) with all the models for that particular release.

Let me know if this is clear or you have more questions.

BR/Luis



On May 7, 2019, at 9:25 AM, rohan julka <rohanjulka19@...> wrote:

Isn’t this what is expected from this internship [1] ?

[1] https://github.com/YangModels/yang/pull/558

Regards,
Rohan
From: Luis Gomez
Sent: 07 May 2019 04:51
To: rohan julka
Cc: integration-dev@...
Subject: Re: [integration-dev] Query regarding internship project - YangModels Publish

Yes there is some initial work done by the beginning of the year but I do not think this has been completed, therefore this internship.

A few more pointers for this internship:

- In ODL, Yang models are used to define REST interface (e.g. RESTCONF). You can find an overview of OpenDaylight in https://docs.opendaylight.org/en/stable-neon/getting-started-guide/index.html and you can also check how ODL projects define their API in the code in this link: https://docs.opendaylight.org/en/stable-neon/developer-guide/developing-apps-on-the-opendaylight-controller.html

- You can find more context of this work in this recording: http://r.lfnetworking.org/?prefix=lfn-zoom/OpenDaylight/TWS/2018-07-23%2010.08.44%20OpenDaylight%20Technical%20Work%20Stream%20Call%20522266747/ and this mail: https://lists.opendaylight.org/pipermail/integration-dev/2018-August/012378.html 

- Finally the work you mention is collected in this issue: https://github.com/YangModels/yang/issues/556, so it would be good to reuse as much as possible.

BR/Luis



On May 6, 2019, at 1:17 PM, rohan julka <rohanjulka19@...> wrote:

Hi ,
I will be applying for OpenDaylight internship[1] for project Yang Models Publish . It seems like that it has already been implemented here [2].  


[1] https://wiki.opendaylight.org/view/Interns/Projects
[2] https://github.com/YangModels/yang/pull/557/files

Regards ,
Rohan
_______________________________________________
integration-dev mailing list
integration-dev@...
https://lists.opendaylight.org/mailman/listinfo/integration-dev



Robert Varga
 

On 10/05/2019 18:32, Luis Gomez wrote:

Now if the models needs to be pushed in the master folder every week ,
the sync script https://github.com/YangModels/yang/pull/557 will not
work as it requires the auto release path which already contains the
submodules of all the repos updated. So we can either submodule each
ODL project and pass it to the sync script or create a script which
will go through each ODL project repo , get the yang models and push
all the models to yang models repo.
Yeah, I am not sure current AR submodules cover all projects we want in
ODL, specially the MRI projects that are not part of AR. Best place to
see project and version for a given branch is this file in distribution:

https://git.opendaylight.org/gerrit/gitweb?p=integration/distribution.git;a=blob;f=artifacts/upstream/properties/pom.xml;h=e4c41bbb29400d2e53166b8cf1e19d4c4baa4067;hb=HEAD
Actually, we should not be relying on ODL source code repositories to
find the models.

Models should be extracted from .jars which are part of a built
distribution (.zip/.tar.gz) -- hence whatever job publishes them should
react to integration/distribution being updated.

There are two main reasons for that:
- we only want to publish models we actually ship, not various testing
models etc.
- YANG files packaged in jars have their names normalized to YANG
specification (i.e. file name matches module/submodule declaration,
revision is part of name) -- that is not how we store them in git repos

Regards,
Robert


Robert Varga
 

On 11/05/2019 12:08, Robert Varga wrote:
Yeah, I am not sure current AR submodules cover all projects we want in
ODL, specially the MRI projects that are not part of AR. Best place to
see project and version for a given branch is this file in distribution:

https://git.opendaylight.org/gerrit/gitweb?p=integration/distribution.git;a=blob;f=artifacts/upstream/properties/pom.xml;h=e4c41bbb29400d2e53166b8cf1e19d4c4baa4067;hb=HEAD
Actually, we should not be relying on ODL source code repositories to
find the models.

Models should be extracted from .jars which are part of a built
distribution (.zip/.tar.gz) -- hence whatever job publishes them should
react to integration/distribution being updated.

There are two main reasons for that:
- we only want to publish models we actually ship, not various testing
models etc.
- YANG files packaged in jars have their names normalized to YANG
specification (i.e. file name matches module/submodule declaration,
revision is part of name) -- that is not how we store them in git repos
And yes, Michael's patches are doing the wrong thing by sifting through
our source code repositories -- they'd pick up things like CSIT driver
models and mdsal-binding-test-model (which supports MD-SAL unit tests)
-- provided mdsal were a part of autorelease, which it is not.

The script to extract the modules from an unpacked distribution already
lives here:
https://github.com/opendaylight/integration-distribution/blob/master/scripts/extract_modules.sh

Regards,
Robert


Luis Gomez
 


On May 11, 2019, at 3:17 AM, Robert Varga <nite@...> wrote:

On 11/05/2019 12:08, Robert Varga wrote:
Yeah, I am not sure current AR submodules cover all projects we want in
ODL, specially the MRI projects that are not part of AR. Best place to
see project and version for a given branch is this file in distribution:

https://git.opendaylight.org/gerrit/gitweb?p=integration/distribution.git;a=blob;f=artifacts/upstream/properties/pom.xml;h=e4c41bbb29400d2e53166b8cf1e19d4c4baa4067;hb=HEAD
Actually, we should not be relying on ODL source code repositories to
find the models.

Models should be extracted from .jars which are part of a built
distribution (.zip/.tar.gz) -- hence whatever job publishes them should
react to integration/distribution being updated.

There are two main reasons for that:
- we only want to publish models we actually ship, not various testing
models etc.
- YANG files packaged in jars have their names normalized to YANG
specification (i.e. file name matches module/submodule declaration,
revision is part of name) -- that is not how we store them in git repos

And yes, Michael's patches are doing the wrong thing by sifting through
our source code repositories -- they'd pick up things like CSIT driver
models and mdsal-binding-test-model (which supports MD-SAL unit tests)
-- provided mdsal were a part of autorelease, which it is not.

The script to extract the modules from an unpacked distribution already
lives here:
https://github.com/opendaylight/integration-distribution/blob/master/scripts/extract_modules.sh

Now I remember this script, I wonder why this was not used in https://github.com/YangModels/yang/issues/556. Anyway, we will evaluate all available scripts during the internship project.


Regards,
Robert