[controller-dev] consistent data store for SR?

Ed Warnicke (eaw) eaw at cisco.com
Thu Jul 11 15:47:44 UTC 2013


Infinispan actually has the capacity to persist its caches:

https://docs.jboss.org/author/display/ISPN/Write-Through+And+Write-Behind+Caching

So perhaps the thing to do is to augment the clustering services API to allow folks to mark some data
as 'persistent' and then persist that via whatever clustering store is provided.

Thoughts?

Ed
On Jul 8, 2013, at 5:01 PM, Colin Dixon <ckd at us.ibm.com<mailto:ckd at us.ibm.com>> wrote:


So, right now, we don't have any way to persist state in Infinispan.

One idea would be to write a new module to the current clustering services APIs that would write snapshots of the key-value stores to disk locally and re-read them on reboot. This would work perfectly without clustering and with clustering it would still work OK as long as each instance checked to see if anyone else was up sharing that cache (key-value store) and if not read it's local copy in and pushed it out. There are some corner cases when multiple instances come back at the same time and all try to load their copy of the data in at the same time, but there might be pretty easy solutions in practice.

Alternatively, you could implement a simple JDBC module that talked to an external DBMS and exported either a new persistent storage service API or just offered the same clustering services API, but provided different semantics. If you do the latter, you'd have to make sure that modules could select which implementation they wanted when they asked for the clustering services API though.

Hopefully this helps a bit.

Are there other options that people are thinking about as well?

--Colin

David Lenrow <david.lenrow at plexxi.com<mailto:david.lenrow at plexxi.com>> wrote on 07/08/2013 12:56:32 PM:
> From: David Lenrow <david.lenrow at plexxi.com<mailto:david.lenrow at plexxi.com>>
> To: Colin Dixon/Austin/IBM at IBMUS
> Date: 07/08/2013 12:59 PM
> Subject: Re: [controller-dev] consistent data store for SR?
>
> Hi Colin,
> The module we're thinking of building/contributing is in a bit of a
> grey area. It's not real-time, relatively low rate of change, but
> needs to be persisted and broadly shared/accessible within
> controller (instances).
>
> If the intention is to have a distributed DBMS sitting behind the
> Infinispan caches to persist across reboots, etc. I might be tempted
> to use this. Any restrictions on volume of data or rate of change
> that make sense to target this service?
>
> If the infinispan state does not persist, then I would need an
> alternative distributed persistence service.
>
> Any thoughts/advice? Can we overload the infinispan service to get
> persistent/consistent behavior from non-realtime state?
>
>
> On Mon, Jul 8, 2013 at 12:15 PM, Colin Dixon <ckd at us.ibm.com<mailto:ckd at us.ibm.com>> wrote:
> To add to what Madhu said, I think that the community would welcome
> a contribution that added this functionality.
>
> Right now, we have more of a real-time, consistent key-value store
> which is provided through the clustering services API and backed by
> Infinispan, but it probably makes sense to have another interface
> which is designed less for real-time cluster synchronization and
> more for long-term, stable storage but perhaps angled less toward
> real-time synchronization.
>
> My guess is that this could be added without too much effort by
> leveraging an external datastore of some kind and maybe using JDBC.
> If you're interested, maybe you could start something up?
>
> --Colin
>
> controller-dev-bounces at lists.opendaylight.org<mailto:controller-dev-bounces at lists.opendaylight.org> wrote on 07/05/2013 11:12:03 AM:
> > From: "Madhu Venugopal (vmadhu)" <vmadhu at cisco.com<mailto:vmadhu at cisco.com>>
> > To: David Lenrow <david.lenrow at plexxi.com<mailto:david.lenrow at plexxi.com>>, "controller-
> > dev at lists.opendaylight.org<mailto:dev at lists.opendaylight.org>" <controller-dev at lists.opendaylight.org<mailto:controller-dev at lists.opendaylight.org>>
> > Date: 07/05/2013 11:12 AM
> > Subject: Re: [controller-dev] consistent data store for SR?
> > Sent by: controller-dev-bounces at lists.opendaylight.org<mailto:controller-dev-bounces at lists.opendaylight.org>
>
> >
> > Yes. The existing controller modules uses the simplistic approach of
> > object serialization to persist Configuration states.
> > The next steps that we are working on is to provided the centralized
> > configuration service provided by the platform to persist states on
> > behalf of the modules.
> > The actual Service implementation will abstract the actual mechanism
> > used for the persistence.
> > We can initially start with this implementation following the
> > serialization approach and this can be replaced with any other
> > database mechanism without
> > any impact on the functional modules. We are following similar
> > approaches to other functional modules as well (please refer to
> > HostTracker as an example).
> >
> > -Madhu
> >
> > From: David Lenrow <david.lenrow at plexxi.com<mailto:david.lenrow at plexxi.com>>
> > Date: Wednesday, July 3, 2013 9:35 AM
> > To: "controller-dev at lists.opendaylight.org<mailto:controller-dev at lists.opendaylight.org>" <controller-
> > dev at lists.opendaylight.org<mailto:dev at lists.opendaylight.org>>
> > Subject: [controller-dev] consistent data store for SR?
> >
> > I am trying to understand the plan of record for delivering
> > distributed database kinds of functionality. The wiki has the
> > "OpenDayLight State Database" (OSD) proposal. This describes
> > development of a module to provide "a hub to access/share/persist
> > state from the other components in the system." I'm hoping such a
> > thing is included in the deliverables for simultaneous release plan.
> > If this is documented somewhere that I have overlooked, please
> provide a link.
> >
> > I see that the existing controller modules persist state by Java
> > object serialization which doesn't provide all the nice attributes
> > of the proposed OSD. Is there an SR milestone when existing
> > persistence code in the repo will be updated to use OSD APIs? It
> > would help to know when/if one can expect the platform to provide
> > this kind of service.
> > --
> > Cheers,
> > -drl
> > ----------------------
> > Dave Lenrow
> > Director, Product Strategy
> > Plexxi
> > 617 329 1861
> > _______________________________________________
> > controller-dev mailing list
> > controller-dev at lists.opendaylight.org<mailto:controller-dev at lists.opendaylight.org>
> > https://lists.opendaylight.org/mailman/listinfo/controller-dev
>

>
> --
> Cheers,
> -drl
> ----------------------
> Dave Lenrow
> Director, Product Strategy
> Plexxi
> 617 329 1861

_______________________________________________
controller-dev mailing list
controller-dev at lists.opendaylight.org<mailto:controller-dev at lists.opendaylight.org>
https://lists.opendaylight.org/mailman/listinfo/controller-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendaylight.org/pipermail/controller-dev/attachments/20130711/732cc2af/attachment.html>


More information about the controller-dev mailing list