[tsc][release] Configure Smart Proxy between ODL Nexus and Central


Anil Belur
 

Greetings all,

Sonatype has updated the issue that was opened Maven Central [1.] on the missing staging repositories and made some recommendations. A large number of staging repositories along with dup artifacts is making Central instable. Therefore they have recommended a workaround to configure the proxy between ODL Nexus and Maven Central [2.], that will inform and publish artifacts downstream when they are created on ODL Nexus.

Going forward, we'll need to configure the smart proxy on our the environment to sync and publish artifacts onto Central. Also, we'll need to ensure this is done on priority so that ODL's staging repositories for the upcoming releases don't get purged.

This will require tweaking of our existing jobs/config once the sync is setup. Below is the list of changes I've identified.

1. Maven stage jobs (for all SM projects) and AR jobs would stage only on ODL Nexus, and therefore the relevant bits to sign/upload to Central might be skipped or disabled. And the staging-repo.txt.gz files are updated with only a ODL Nexus repos.
2. Self-release jobs might require a few updates/tweaking to ensure Central repo promotion is skipped or disabled.
3. Deploy required configuration changes to setup ODL Nexus and Central smart proxy.
4. Purge all the staging repositories on Maven Central (excluding the repositories that are approved for releases).
5. Sync and test ODL Nexus and Central proxy

[1.] https://issues.sonatype.org/browse/MVNCENTRAL-4961
[2.] https://central.sonatype.org/pages/central-sync-with-nexus-smart-proxy.html
[3.] https://lists.opendaylight.org/pipermail/tsc/2019-June/011628.html

Echoing the response I've received from Sonatype:

"

Recent instability with OSSRH has warranted further investigation into this issue. As of this moment, it appears that approximately 45% of all closed staging repositories on oss.sonatype.org belong to opendaylight.org. As Joel mentioned above, large numbers of unused staging repositories can severely impact the performance of oss.sonatype.org. Other users are now beginning to notice and are issuing complaints about service degradation requiring that we take action on this immediately.

We request that you remove excessive (more than 10) staging repositories as soon as possible, otherwise we may be forced to drop these staging repositories and prohibit access until your development and deployment process is modified.  

In situations like yours, we recommend syncing releases directly from your public Nexus instance (nexus.opendaylight.org) to Central.  This would seemingly allow your current release process to be maintained and significantly reduce load on our servers. Please see the following URL for instructions on how this is configured:

"


Anil Belur
 

 

On Wed, Aug 28, 2019 at 10:06 AM Anil Belur <abelur@...> wrote:

Greetings all,

Sonatype has updated the issue that was opened Maven Central [1.] on the missing staging repositories and made some recommendations. A large number of staging repositories along with dup artifacts is making Central instable. Therefore they have recommended a workaround to configure the proxy between ODL Nexus and Maven Central [2.], that will inform and publish artifacts downstream when they are created on ODL Nexus.

Going forward, we'll need to configure the smart proxy on our the environment to sync and publish artifacts onto Central. Also, we'll need to ensure this is done on priority so that ODL's staging repositories for the upcoming releases don't get purged.

This will require tweaking of our existing jobs/config once the sync is setup. Below is the list of changes I've identified.

1. Maven stage jobs (for all SM projects) and AR jobs would stage only on ODL Nexus, and therefore the relevant bits to sign/upload to Central might be skipped or disabled. And the staging-repo.txt.gz files are updated with only a ODL Nexus repos.
2. Self-release jobs might require a few updates/tweaking to ensure Central repo promotion is skipped or disabled.
3. Deploy required configuration changes to setup ODL Nexus and Central smart proxy.
4. Purge all the staging repositories on Maven Central (excluding the repositories that are approved for releases).
5. Sync and test ODL Nexus and Central proxy

[1.] https://issues.sonatype.org/browse/MVNCENTRAL-4961
[2.] https://central.sonatype.org/pages/central-sync-with-nexus-smart-proxy.html
[3.] https://lists.opendaylight.org/pipermail/tsc/2019-June/011628.html

Echoing the response I've received from Sonatype:

"

Recent instability with OSSRH has warranted further investigation into this issue. As of this moment, it appears that approximately 45% of all closed staging repositories on oss.sonatype.org belong to opendaylight.org. As Joel mentioned above, large numbers of unused staging repositories can severely impact the performance of oss.sonatype.org. Other users are now beginning to notice and are issuing complaints about service degradation requiring that we take action on this immediately.

We request that you remove excessive (more than 10) staging repositories as soon as possible, otherwise we may be forced to drop these staging repositories and prohibit access until your development and deployment process is modified.  

In situations like yours, we recommend syncing releases directly from your public Nexus instance (nexus.opendaylight.org) to Central.  This would seemingly allow your current release process to be maintained and significantly reduce load on our servers. Please see the following URL for instructions on how this is configured:

"


Hello Abhijit: 

PS include the topic for tomorrow's TSC meeting agenda.

Thanks,
Anil 


Abhijit Kumbhare
 

Sure. Tomorrow is going to be a little busy day as we also need to go through the LFN Budget 2020 spreadsheet that we have been asked to fill out.


On Tue, Aug 27, 2019 at 11:35 PM Anil Belur <abelur@...> wrote:
 

On Wed, Aug 28, 2019 at 10:06 AM Anil Belur <abelur@...> wrote:

Greetings all,

Sonatype has updated the issue that was opened Maven Central [1.] on the missing staging repositories and made some recommendations. A large number of staging repositories along with dup artifacts is making Central instable. Therefore they have recommended a workaround to configure the proxy between ODL Nexus and Maven Central [2.], that will inform and publish artifacts downstream when they are created on ODL Nexus.

Going forward, we'll need to configure the smart proxy on our the environment to sync and publish artifacts onto Central. Also, we'll need to ensure this is done on priority so that ODL's staging repositories for the upcoming releases don't get purged.

This will require tweaking of our existing jobs/config once the sync is setup. Below is the list of changes I've identified.

1. Maven stage jobs (for all SM projects) and AR jobs would stage only on ODL Nexus, and therefore the relevant bits to sign/upload to Central might be skipped or disabled. And the staging-repo.txt.gz files are updated with only a ODL Nexus repos.
2. Self-release jobs might require a few updates/tweaking to ensure Central repo promotion is skipped or disabled.
3. Deploy required configuration changes to setup ODL Nexus and Central smart proxy.
4. Purge all the staging repositories on Maven Central (excluding the repositories that are approved for releases).
5. Sync and test ODL Nexus and Central proxy

[1.] https://issues.sonatype.org/browse/MVNCENTRAL-4961
[2.] https://central.sonatype.org/pages/central-sync-with-nexus-smart-proxy.html
[3.] https://lists.opendaylight.org/pipermail/tsc/2019-June/011628.html

Echoing the response I've received from Sonatype:

"

Recent instability with OSSRH has warranted further investigation into this issue. As of this moment, it appears that approximately 45% of all closed staging repositories on oss.sonatype.org belong to opendaylight.org. As Joel mentioned above, large numbers of unused staging repositories can severely impact the performance of oss.sonatype.org. Other users are now beginning to notice and are issuing complaints about service degradation requiring that we take action on this immediately.

We request that you remove excessive (more than 10) staging repositories as soon as possible, otherwise we may be forced to drop these staging repositories and prohibit access until your development and deployment process is modified.  

In situations like yours, we recommend syncing releases directly from your public Nexus instance (nexus.opendaylight.org) to Central.  This would seemingly allow your current release process to be maintained and significantly reduce load on our servers. Please see the following URL for instructions on how this is configured:

"


Hello Abhijit: 

PS include the topic for tomorrow's TSC meeting agenda.

Thanks,
Anil 


Robert Varga
 

On 28/08/2019 06:36, Anil Belur wrote:
Greetings all,

Sonatype has updated the issue that was opened Maven Central [1.] on the
missing staging repositories and made some recommendations. A large
number of staging repositories along with dup artifacts is making
Central instable. Therefore they have recommended a workaround to
configure the proxy between ODL Nexus and Maven Central [2.], that will
inform and publish artifacts downstream when they are created on ODL Nexus.

Going forward, we'll need to configure the smart proxy on our the
environment to sync and publish artifacts onto Central. Also, we'll need
to ensure this is done on priority so that ODL's staging repositories
for the upcoming releases don't get purged.
Hello again,

I would like to note that this activity has been successfully completed
(thanks Anil and the Sonatype folks).

There is one unforeseen side-effect of this: all of our artifacts ever
released (well, at least all the way back to Helium release) are now
available on Maven Central -- i.e.
https://search.maven.org/search?q=g:org.opendaylight.controller%20AND%20a:sal-binding-api&core=gav

The immediate upshot of this is that we have javadocs for all artifacts
also available through javadoc.io, for example:

https://static.javadoc.io/org.opendaylight.controller/sal-binding-api/1.7.4/index.html?overview-summary.html

Regards,
Robert


Anil Belur
 


On Mon, Sep 16, 2019 at 1:07 AM Robert Varga <nite@...> wrote:
On 28/08/2019 06:36, Anil Belur wrote:
> Greetings all,
>
> Sonatype has updated the issue that was opened Maven Central [1.] on the
> missing staging repositories and made some recommendations. A large
> number of staging repositories along with dup artifacts is making
> Central instable. Therefore they have recommended a workaround to
> configure the proxy between ODL Nexus and Maven Central [2.], that will
> inform and publish artifacts downstream when they are created on ODL Nexus.
>
> Going forward, we'll need to configure the smart proxy on our the
> environment to sync and publish artifacts onto Central. Also, we'll need
> to ensure this is done on priority so that ODL's staging repositories
> for the upcoming releases don't get purged.

Hello again,

I would like to note that this activity has been successfully completed
(thanks Anil and the Sonatype folks).

There is one unforeseen side-effect of this: all of our artifacts ever
released (well, at least all the way back to Helium release) are now
available on Maven Central -- i.e.
https://search.maven.org/search?q=g:org.opendaylight.controller%20AND%20a:sal-binding-api&core=gav

The immediate upshot of this is that we have javadocs for all artifacts
also available through javadoc.io, for example:

https://static.javadoc.io/org.opendaylight.controller/sal-binding-api/1.7.4/index.html?overview-summary.html

Regards,
Robert


Hello Robert, 

That's because we have forced a complete re-sync of everything in opendaylight-releases, just to make sure the missing release are all available. I believe I've bought it up earlier in the TSC call and also on the IRC discussion.

Thanks,
Anil