Re: Git workflows
Guillaume Lambert
Hi all
I am quite in line with what
Robert wrote.
It is worth to notice that ONAP already gave a shot on migrating its CI to GitLab
and this was a subsequent effort. You can find more details at these URLs.
https://wiki.onap.org/display/DW/ONAP+CD+on+Gitlab
https://wiki.onap.org/display/DW/Daily+Deployments+and+gating
This is a bit different but it can give an idea of what dealing with such a migration implies.
Hope this helps
Best Regards
Guillaume
De : TSC@... <TSC@...> de la part de Robert Varga <nite@...>
Envoyé : mardi 9 août 2022 22:58
À : tsc@...
Objet : [OpenDaylight TSC] Git workflows
Envoyé : mardi 9 août 2022 22:58
À : tsc@...
Objet : [OpenDaylight TSC] Git workflows
Hello,
I am slowly catching up on things from last month. One item is the
subject of Github workflows.
There are a number of unresolved issues, some of which may be my
ignorance of the outside world (in which case I would *love* to be
proven wrong). Here is the list:
1. Support for multiple branches
================================
It is OpenDaylight policy to support up to 3 branches at any given time
for any MSI project. For MRI projects, that number gets to 4 for periods
last 2-5 months -- as is the case for YANG tools right now, we have:
- yangtools-7.0.x for 2022.03 Phosphorus security support
- yangtools-8.0.x for 2022.06 Sulfur
- yangtools-9.0.x for 2022.09 Chlorine
- yangtools-master for 2023.03 Argon
As far as I know, Github does not provide the equivalent of Gerrit
cherry-picks out of the box. That certainly was the case ~5 years when I
investigated this more deeply.
The crux of the issue seems to be Change-ID and its tie-in with GH PRs.
I was told by Andy Grimberg this is nigh impossible to reconcile.
Change-ID is critical for cross-referencing commits, because equivalent
patches can look very differently on each supported branch.
That having been said, I do believe this is fixable by automation, e.g.
having a bot assign Change-IDs for a PR and squashing each PR into a
single patch -- which then can be projected to Gerrit, allowing for
migration. I am not aware of such a bot existing, so I track this as
something would have to be contributed.
2. Permissions
==============
Github is a system external to LF. As such, I do not think there is
infrastructure present to project each project's INFO.yaml into Github
permissions. AFAICT the only existing thing is the 'OpenDaylight
project', which is an all-or-nothing thing. That is something LF IT has
to tackle before we consider migrating.
3. Verification
===============
Our current infrastructure is tied to Jenkins. A switch to GH requires
that a PR triggers the appropriate jobs in Jenkins. Unless we are
talking a straight-up move to GH Actions, we need point 1. to be solved
and drive verification projected from Gerrit back to GH. If GH actions
are in the picture, at least maven-verify need to be migrated. Again,
this needs a community contribution.
Now, I am not a nay-sayer, but we have a sh*tload of things going and
migrating it requires some serious man-power and for those uninitiated,
I like to quote Thanh: "Getting Eclipse build where it is was a twelve
month effort for three dedicated people" (IIRC). We no longer have Thanh
and all his dedication and experience.
I have used this quote when someone proposed moving to Gradle. Moving to
GH workflows is harder.
If we are to tackle this, we need to solve above problems in order: 2,
1, 3. I will lend my support to anyone seriously committed to this
undertaking.
At the end of the day, this is not impossible. The OpenJDK community has
executed a full transition from custom Mercurial workflows (webrev et
al.) to GitHub PRs -- but that transition includes a metric ton of
automation which had to be written from scratch. We as a community are
struggling to get to Jenkins Pipelines, which is dead simple in comparison.
So, Venkat, as the one proposing this change, are you in a position and
willing to drive it to completion?
Regards,
Robert
I am slowly catching up on things from last month. One item is the
subject of Github workflows.
There are a number of unresolved issues, some of which may be my
ignorance of the outside world (in which case I would *love* to be
proven wrong). Here is the list:
1. Support for multiple branches
================================
It is OpenDaylight policy to support up to 3 branches at any given time
for any MSI project. For MRI projects, that number gets to 4 for periods
last 2-5 months -- as is the case for YANG tools right now, we have:
- yangtools-7.0.x for 2022.03 Phosphorus security support
- yangtools-8.0.x for 2022.06 Sulfur
- yangtools-9.0.x for 2022.09 Chlorine
- yangtools-master for 2023.03 Argon
As far as I know, Github does not provide the equivalent of Gerrit
cherry-picks out of the box. That certainly was the case ~5 years when I
investigated this more deeply.
The crux of the issue seems to be Change-ID and its tie-in with GH PRs.
I was told by Andy Grimberg this is nigh impossible to reconcile.
Change-ID is critical for cross-referencing commits, because equivalent
patches can look very differently on each supported branch.
That having been said, I do believe this is fixable by automation, e.g.
having a bot assign Change-IDs for a PR and squashing each PR into a
single patch -- which then can be projected to Gerrit, allowing for
migration. I am not aware of such a bot existing, so I track this as
something would have to be contributed.
2. Permissions
==============
Github is a system external to LF. As such, I do not think there is
infrastructure present to project each project's INFO.yaml into Github
permissions. AFAICT the only existing thing is the 'OpenDaylight
project', which is an all-or-nothing thing. That is something LF IT has
to tackle before we consider migrating.
3. Verification
===============
Our current infrastructure is tied to Jenkins. A switch to GH requires
that a PR triggers the appropriate jobs in Jenkins. Unless we are
talking a straight-up move to GH Actions, we need point 1. to be solved
and drive verification projected from Gerrit back to GH. If GH actions
are in the picture, at least maven-verify need to be migrated. Again,
this needs a community contribution.
Now, I am not a nay-sayer, but we have a sh*tload of things going and
migrating it requires some serious man-power and for those uninitiated,
I like to quote Thanh: "Getting Eclipse build where it is was a twelve
month effort for three dedicated people" (IIRC). We no longer have Thanh
and all his dedication and experience.
I have used this quote when someone proposed moving to Gradle. Moving to
GH workflows is harder.
If we are to tackle this, we need to solve above problems in order: 2,
1, 3. I will lend my support to anyone seriously committed to this
undertaking.
At the end of the day, this is not impossible. The OpenJDK community has
executed a full transition from custom Mercurial workflows (webrev et
al.) to GitHub PRs -- but that transition includes a metric ton of
automation which had to be written from scratch. We as a community are
struggling to get to Jenkins Pipelines, which is dead simple in comparison.
So, Venkat, as the one proposing this change, are you in a position and
willing to drive it to completion?
Regards,
Robert
_________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.