Re: [integration-dev] [release] [it-infrastructure-alerts][notice] ODL Jenkins maintenance window - 1600-1800 PST, Mar 01, 2020.


Andrew Grimberg
 

Ok, we should be good now. I've updated the plugin, verified that builds
of ci-management-cfg-merge should _not_ destroy the cloud config again
and released the build queue.

We should be good now. Sorry for all the churn on this. We're working on
developing a new method for doing the community manageable parts of
Jenkins to be less brittle. Our cloud management scripting while it
works is unfortunately subject to getting messed up by API changes in
the plugin.

-Andy-

On 2020-03-03 10:09, Andrew Grimberg via Lists.Opendaylight.Org wrote:
We're going to be taking another brief outage on the Jenkins node.

While we got things up and working again it all came to a halt when the
regularly schedule ci-management-cfg-merge job fired and wiped out the
cloud configuration.

It looks like the version of the OpenStack plugin we're running is not
compatible with the latest version of global-jjb. We're going to change
the plugin to the version that I just verified works on the ODL sandbox.

-Andy-

On 2020-03-02 23:55, Abhijit Kumbhare wrote:
Thanks for the update. Hopefully gets resolved soon.

On Mon, Mar 2, 2020 at 11:14 PM Anil Belur <abelur@...
<mailto:abelur@...>> wrote:

Hello all,

The maintenance work is now complete. We've had to copy all plugins
from sandbox over to releng, since resolving the dependencies
manually is not straight forward and Jenkins plugin management does
not handle dependencies as well as packaging. We have an existing
backup of the plugin directory. Going forward, if we have any
missing plugins on the production/releng that will need to be installed.

Note: we are seeing a metadata issue on the cloud, when the nodes
are coming online, which is causing the Jenkins jobs to fail. I'm
working with the cloud provider to get this resolved. 

Apologies for any inconvenience. 

Thanks,
Anil

On Thu, Feb 27, 2020 at 1:35 PM Anil Belur via
Lists.Opendaylight.Org <http://Lists.Opendaylight.Org>
<abelur=linuxfoundation.org@...
<mailto:linuxfoundation.org@...>> wrote:

What: LF will perform maintenance on ODL Jenkins servers to
upgrade the plugins to the latest.

When: 1600-1800 PST, Sunday, Mar 01, 2020

Why: The latest OpenStack API requires OpenStack-clouds plugin
updated and also many other plugins outdated and require to be
updated. 
 
Impact: All ODL Jenkins (sandbox/releng) may be inaccessible
during this time and
Jenkins will be put in shutdown mode before the window starts
and any long-running Jenkins jobs _will_ be canceled if they
don't complete before the start of the window.

Notices will be posted to the mailing lists and in #opendaylight
on Freenode at the start and end of the maintenance.

Thanks,
Anil Belur






--
Andrew J Grimberg
Manager Release Engineering
The Linux Foundation

Join TSC@lists.opendaylight.org to automatically receive all group messages.