Re: [E] Re: [integration-dev] integration/distribution version issues


Luis Gomez
 

BTW below I meant "consolidate back to single distribution".

On Nov 17, 2021, at 12:01 PM, Luis Gomez via lists.opendaylight.org <ecelgp=gmail.com@...> wrote:

I think the part that is confusing is to have 2 distros so different (life-cycle, release, tagging, etc) in the same repo, I remember we did this long time back because we wanted to avoid having a new repo for just 1 file (opendaylight/pom.xml) but since then it has been only problematic and complex (e.g. different jobs and processes for different folders in the repo).

Ideally we should separate these 2 in its own repo, jobs, etc. Or better, consolidate back to single repo if we can get TPCE to be a managed project. For your question, the only reason we need 2 distribution releases is because there is one Self Managed projects that actively participates in the ODL distribution. Also see more details on why we are releasing 2 distros here: https://docs.opendaylight.org/en/stable-phosphorus/release-process/managed-release.html, however all this managed vs self-managed release was invented in a time where we had many projects in ODL and for me this does not make sense anymore.

BR/Luis


On Nov 17, 2021, at 10:12 AM, Ha, Sangwook <sangwook.ha@...> wrote:

Regarding the version mismatch for the common release, what I meant was for the karaf version in 'opendaylight/pom.xml' and the other versions - for example, release tag '14.3.0' would have the karaf version of '0.14.3' in 'opendaylight/pom.xml' but the rest has '0.14.4'. So both '14.3.0' and 'release/silicon-sr4', when released, will have '0.14.4' as the version for all the artifacts, at least when looking at the repository.

Thanks,
Sangwook

On Wed, Nov 17, 2021 at 8:42 AM Sangwook Ha via lists.opendaylight.org<sangwook.ha=verizon.com@...> wrote:
Looking at the release & version bump cycle, some of the steps may be simplified, and hopefully automated:

- For each release of managed projects, release & version bump, are done separately a few days apart - is there any reason why this cannot be done together?
- For 'opendaylight/pom.xml' there are multiple manual steps: can steps 2 & 3 be merged & done automatically?
1) activate profiles for self-managed projects
2) release
3) update versions and deactivate profiles for self-managed projects (sometimes the version bump is done in two separate steps: the artifact/karaf & self-managed project)

And there are two different types of release tags - one for managed projects (e.g. release/silicon-sr3) and common release (14.3.0). The former does not update 'opendaylight/pom.xml' but the latter updates all the POM files. And this is confusing because all the versions except for 'opendaylight/pom.xml' have been bumped up by the time the release is made, and they have a version ahead of what the label says (e.g. for the tag '14.3.0' all the versions except for 'opendaylight/pom.xml' is '14.4.0').

Thanks,
Sangwook

On Wed, Nov 17, 2021 at 1:18 AM Robert Varga <nite@...> wrote:
On 17/11/2021 08:26, Luis Gomez wrote:
> I thought this was clear, at least to ODL old folks, the int/dist 
> project holds 2 distributions:

Three, actually.

> - Karaf distribution (karaf/pom.xml) only containing Managed projects is 
> also a Managed project and integrated with autorelease (automatic 
> release & bump).
> - Common distribution (opendaylight/pom.xml) containing Managed and Self 
> Managed projects. This is a Self Managed project and therefore it has to 
> be manually released, bumped, etc, just like any other SM project.

Yes, and therefore the release lifecycle of int/dist is unlike any other 
project I have come across.

> AFAIR the sanity test you are pointing out is the only test that uses 
> the common distribution, all of our CSIT uses Karaf distribution. I hope 
> this explains.

Right-o, but unfortunately you are explaining something completely 
off-topic, so let me try to reiterate.

1. Ever since the dawn of autorelease MSI projects have agreed to bump 
the minor version, i.e. 1.2.0 -> 1.3.0

2. On Feb 22 this year, Anil branched stable/silicon, correctly bumping 
opendaylight/pom.xml from 0.14.0-SNAPSHOT to 0.15.0-SNAPSHOT: 
https://git.opendaylight.org/gerrit/c/integration/distribution/+/95287

3. On Apr 3 this year, you changed the versioning scheme on 
stable/silicon to 14.0.0-SNAPSHOT: 
https://git.opendaylight.org/gerrit/c/integration/distribution/+/95655

4. On Apr 22 this year, Guillaume made a similar change on then-master: 
https://git.opendaylight.org/gerrit/c/integration/distribution/+/95789

5. On Sep 21 this year, Anil branched stable/phosphorus, but unlike all 
the previous times, opendaylight/pom.xml's version was NOT updated: 
https://git.opendaylight.org/gerrit/c/integration/distribution/+/97551

6. On Sep 24 this year, the first successful 
distribution-merge-full-sulfur job run: 
https://s3-logs.opendaylight.org/logs/releng/vex-yul-odl-jenkins-1/distribution-merge-full-sulfur/2/console.log.gz, 
happily doing:

> Deploying the main artifact opendaylight-15.0.0-SNAPSHOT.tar.gz
> Uploading to opendaylight-snapshot: https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/opendaylight/15.0.0-SNAPSHOT/opendaylight-15.0.0-20210924.032546-1083.tar.gz
> Uploaded to opendaylight-snapshot: https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/opendaylight/15.0.0-SNAPSHOT/opendaylight-15.0.0-20210924.032546-1083.tar.gz (266 MB at 28 MB/s)
> Uploading to opendaylight-snapshot: https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/opendaylight/15.0.0-SNAPSHOT/maven-metadata.xml
> Uploaded to opendaylight-snapshot: https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/opendaylight/15.0.0-SNAPSHOT/maven-metadata.xml (982 B at 18 kB/s)

7. On Sep 24 this year, run-of-the mill 
distribution-merge-full-phosphorus ran: 
https://s3-logs.opendaylight.org/logs/releng/vex-yul-odl-jenkins-1/distribution-merge-full-phosphorus/1170/console.log.gz, 
happily doing this:

> Deploying the main artifact opendaylight-15.0.0-SNAPSHOT.tar.gz
> Uploading to opendaylight-snapshot: https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/opendaylight/15.0.0-SNAPSHOT/opendaylight-15.0.0-20210924.050736-1084.tar.gz
> Uploaded to opendaylight-snapshot: https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/opendaylight/15.0.0-SNAPSHOT/opendaylight-15.0.0-20210924.050736-1084.tar.gz (266 MB at 28 MB/s)
> Uploading to opendaylight-snapshot: https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/opendaylight/15.0.0-SNAPSHOT/maven-metadata.xml
> Uploaded to opendaylight-snapshot: https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/opendaylight/15.0.0-SNAPSHOT/maven-metadata.xml (982 B at 20 kB/s)

8. This continued for quite some time, i.e. the contents of 
opendaylight-15.0.0-SNAPSHOT flip-flopped between Sulfur and Phosphorus

9. On Oct 24 distribution-merge-full-phosphorus started publishing 
opendaylight-15.1.0-SNAPSHOT: 
https://s3-logs.opendaylight.org/logs/releng/vex-yul-odl-jenkins-1/distribution-merge-full-phosphorus/1322/console.log.gz

10. On Oct 24 distribution-merge-full-sulfur started failing: 
https://s3-logs.opendaylight.org/logs/releng/vex-yul-odl-jenkins-1/distribution-merge-full-sulfur/149/console.log.gz

11. On Nov 13 the last published opendaylight-15.0.0-SNAPSHOT expired in 
Nexus

12. On Nov 14 distribution-merge-full-sulfur started failing: 
https://s3-logs.opendaylight.org/logs/releng/vex-yul-odl-jenkins-1/distribution-merge-full-sulfur/275/console.log.gz 
with

> ERROR: Failed to parse POMs
> hudson.remoting.ProxyException: hudson.maven.MavenModuleSetBuild$MavenExecutionException: org.apache.maven.project.ProjectBuildingException: Some problems were encountered while processing the POMs:
> [FATAL] Non-resolvable parent POM for org.opendaylight.integration:opendaylight:15.0.0-SNAPSHOT: Could not find artifact org.opendaylight.integration:karaf:pom:0.15.0-SNAPSHOT in opendaylight-snapshot (https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/) and 'parent.relativePath' points at no local POM @ line 14, column 13


The bottom line is:

- int/dist setup was busted for almost two months
- job failures have been indicating this clearly for a month
- now finally everything fell apart
- it was a downstream user who detected this mess

I equally hope this explains.

Bye,
Robert

> 
> For more information check this doc: 
> https://docs.opendaylight.org/projects/integration-distribution/en/latest/add-project-distribution.html 
> <https://docs.opendaylight.org/projects/integration-distribution/en/latest/add-project-distribution.html>
> 
> BR/Luis
> 
>> On Nov 16, 2021, at 9:08 PM, Robert Varga <nite@... 
>> <mailto:nite@...>> wrote:
>>
>> On 17/11/2021 00:37, Daniel de la Rosa wrote:
>>> Let me add more Robert and team to make sure that they see this email
>>
>> This boils down to interaction between autorelease and int/dist.
>>
>> autorelease assumes branch cutting involves bumping minor version, 
>> which has been true for all MSI projects since forever.
>>
>> int/dist started violating that assumption by changing versioning 
>> scheme here: 
>> https://git.opendaylight.org/gerrit/c/integration/distribution/+/95655 
>> <https://git.opendaylight.org/gerrit/c/integration/distribution/+/95655>
>>
>> I have no skin in this particular game, sorry.
>>
>> Regards,
>> Robert
>>
>>
>>> On Tue, Nov 16, 2021 at 1:47 PM Sangwook Ha via 
>>> lists.opendaylight.org <http://lists.opendaylight.org> 
>>> <http://lists.opendaylight.org <http://lists.opendaylight.org>> 
>>> <sangwook.ha=verizon.com@... 
>>> <mailto:sangwook.ha=verizon.com@...> 
>>> <mailto:verizon.com@... 
>>> <mailto:verizon.com@...>>> wrote:
>>>    It appears that the versions in
>>>    integration/distribution/opendaylight have not been updated, and
>>>    some Jenkins jobs are failing: e.g.
>>>    openflowplugin-csit-1node-sanity-only-sulfur
>>>    <https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-1node-sanity-only-sulfur/ 
>>> <https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-1node-sanity-only-sulfur/>>
>>>    I submitted two patches to fix up the version issues for Sulfur &
>>>    Silicon - Phosphorus seems okay.
>>>    Shouldn't this be done automatically when it's released? Looks like
>>>    release version bump to remove SNAPSHOT is done automatically but
>>>    SNAPSHOT version is not updated for opendaylight artifact.
>>>    Thanks,
>>>    Sangwook
>>> 
> 




Join integration-dev@lists.opendaylight.org to automatically receive all group messages.