Re: OpenDaylight V2 Driver


Isaku Yamahata <yamahata@...>
 

Hello Vivek. Will you(or someone on behalf of you) attend ONS or
openstack PTG?


On Wed, Feb 14, 2018 at 09:50:14AM +0000,
N Vivekanandan <n.vivekanandan@...> wrote:

Hi Team,

We were contemplating to use v2 Driver for production after noticing that it has huge improvements over the v1 Driver.
Primarily in terms of its stability to use in HA active-active cluster when compared to v1 Driver which
is unstable for use in HA active-active clusters.

After going through available documentation, it looks v2 driver could have a couple of issues that could
occur in production deployments:

a. Journal DB unchecked explosion

- a situation where ODL is permanently disconnected from Openstack , but orchestrations are done in

Openstack Neutron continually successfully during this window. Could this end up with Journal DB overflowing,

And journal threads simply thrashing around between themselves to see if a resource-event can be posted to ODL?
There are two aspect.
- journal thread thrashing
- journal db overflow

When journal thread find connection error(request.exceptions.ConnectionError),
it sleeps with backoff. So it's addressed. The maximum wait time is
hardcoded to 60 secs at the moment.

Regarding to journal db overflow, currently we don't do anything about it.
In theory it can happen. But I'm not sure how seriously it should be
considered and what kind of behavior is desirable. Your input is very helpful.
As long as one of neutron servers can connect to one of ODL servers,
journal entries will be processed. So with HA (both neutron server HA and
ODL HA), it's unlikely that all the ODL servers are disconnected from
all the neutron servers. Or do you have any other scenario?

Actually in the case of plain neutron agent case if the connection
between MQ server-neutorn agent is lost, we have similar situation. MQ
server will accumulate RPC messages.

I think of several options, but at first I'd like to know about
its necessary and scenario.


b. Large number of journal threads (eg., each neutron-api-worker will have one or more ODL Journal thread fired) taking up

CPU cycles competing to address limited PENDING entries in the journal DB. These accesses (including Writes) can actually result in

DB performance drops.

There were ways attempted to restrict the number of journal threads in a process to just 'one' here:

https://review.openstack.org/#/c/444648/

But we are unsure what was concluded as the patch didn't merge.

Is it possible to restrict the journal threads to just one per Neutron-API-Worker process through some other configuration.

Referred slides: https://www.slideshare.net/LCChina/opendaylight-openstack-integration
The patch you pointed was too easy.
The changeset of a13eb005950b189cc4f713722c45706a92d56530 was merged and
the effort stopped there.
Basically the task to check journal db periodically is moved to neutron
worker. which is created one process per neutron server irrelevant of
the number of worker process.

thanks,
--
Isaku Yamahata <isaku.yamahata@...>

Join z.archive.neutron-dev@lists.opendaylight.org to automatically receive all group messages.