1. Z.Archive.Openflowjava-Dev
  2. Topics

Clustering impact on openflowplugin

Moiz Raja
 

The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

Please let me know if you believe this scheme would have problems.

Thanks,
-Moiz

Basheeruddin Ahmed
 

I think more than one component might find the change of mastership (or primary) significant - hence they should be able to register a listener on Clustering Service -- which would notify the change in mastership. Its upto the component how to handle the change in mastership event.

thx
-Basheeruddin
On 06/04/2014 01:10 PM, Moiz Raja wrote:
The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

Please let me know if you believe this scheme would have problems.

Thanks,
-Moiz


-- 
Thx,
-Basheeruddin

Abhishek Kumar
 

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

-Abhishek

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:

The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

Please let me know if you believe this scheme would have problems.

Thanks,
-Moiz

Abhishek Kumar
 

+1
On Jun 4, 2014, at 2:05 PM, Basheeruddin Ahmed <syedbahm@...> wrote:

I think more than one component might find the change of mastership (or primary) significant - hence they should be able to register a listener on Clustering Service -- which would notify the change in mastership. Its upto the component how to handle the change in mastership event.

thx
-Basheeruddin
On 06/04/2014 01:10 PM, Moiz Raja wrote:
The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

Please let me know if you believe this scheme would have problems.

Thanks,
-Moiz


-- 
Thx,
-Basheeruddin

Moiz Raja
 

Basheer,

In general I agree that components should register with ClusteringService to find if a member is up or down. The subtle difference I see is that we need some component that is "shard aware" to make that decision and I do not think openflowplugin should be that "shard aware" component and so I find introducing/using an RPC to be a better choice.

-Moiz

From: Basheeruddin Ahmed (syedbahm)
Sent: Wednesday, June 04, 2014 2:05 PM
To: Moiz Raja (moraja); openflowplugin-dev@...; openflowjava-dev@...
Cc: Abhishek Kumar (abhishk2)
Subject: Re: Clustering impact on openflowplugin

I think more than one component might find the change of mastership (or primary) significant - hence they should be able to register a listener on Clustering Service -- which would notify the change in mastership. Its upto the component how to handle the change in mastership event.

thx
-Basheeruddin
On 06/04/2014 01:10 PM, Moiz Raja wrote:
The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

Please let me know if you believe this scheme would have problems.

Thanks,
-Moiz


-- 
Thx,
-Basheeruddin

Moiz Raja
 

Abhishek,

In the case where a member crashes rpcs will not get unregistered with the system. I think we need to handle that scenario as well - in this scenario obviously the downed node will not be able to unregister the rpcs and not be able to handle a "role change" notification from the switch - which as per the OpenFlow Spec would not be sent to the slave controller anyway. One of the things which distinguishes a OF Master from a Slave controller is that slaves do not receive any asynchronous messages from the switch.

-Moiz

From: Abhishek Kumar (abhishk2)
Sent: Wednesday, June 04, 2014 2:08 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev@...; Basheeruddin Ahmed (syedbahm)
Subject: Re: Clustering impact on openflowplugin

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

-Abhishek

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:

The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

Please let me know if you believe this scheme would have problems.

Thanks,
-Moiz

Abhishek Kumar
 

Yes, but that falls into routed rpc clustering component. Openflow plugin need not worry.

On Jun 4, 2014, at 2:35 PM, Moiz Raja (moraja) <moraja@...> wrote:

Abhishek,

In the case where a member crashes rpcs will not get unregistered with the system. I think we need to handle that scenario as well - in this scenario obviously the downed node will not be able to unregister the rpcs and not be able to handle a "role change" notification from the switch - which as per the OpenFlow Spec would not be sent to the slave controller anyway. One of the things which distinguishes a OF Master from a Slave controller is that slaves do not receive any asynchronous messages from the switch.

-Moiz
From: Abhishek Kumar (abhishk2)
Sent: Wednesday, June 04, 2014 2:08 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev@...; Basheeruddin Ahmed (syedbahm)
Subject: Re: Clustering impact on openflowplugin

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

-Abhishek

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:

The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

Please let me know if you believe this scheme would have problems.

Thanks,
-Moiz

Prasanna Huddar
 

What does member indicate here?

-Prasanna

 

From: openflowplugin-dev-bounces@... [mailto:openflowplugin-dev-bounces@...] On Behalf Of Moiz Raja (moraja)
Sent: Thursday, June 05, 2014 3:05 AM
To: Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

 

Abhishek,

 

In the case where a member crashes rpcs will not get unregistered with the system. I think we need to handle that scenario as well - in this scenario obviously the downed node will not be able to unregister the rpcs and not be able to handle a "role change" notification from the switch - which as per the OpenFlow Spec would not be sent to the slave controller anyway. One of the things which distinguishes a OF Master from a Slave controller is that slaves do not receive any asynchronous messages from the switch.

 

-Moiz

From: Abhishek Kumar (abhishk2)
Sent: Wednesday, June 04, 2014 2:08 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev@...; Basheeruddin Ahmed (syedbahm)
Subject: Re: Clustering impact on openflowplugin

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

 

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

 

-Abhishek

 

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:



The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

 

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

 

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

 

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

 

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

Please let me know if you believe this scheme would have problems.

 

Thanks,

-Moiz

 

Moiz Raja
 

A cluster member.

From: Prasanna Huddar [prasanna.huddar@...]
Sent: Wednesday, June 04, 2014 10:29 PM
To: Moiz Raja (moraja); Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: RE: Clustering impact on openflowplugin

What does member indicate here?

-Prasanna

 

From: openflowplugin-dev-bounces@... [mailto:openflowplugin-dev-bounces@...] On Behalf Of Moiz Raja (moraja)
Sent: Thursday, June 05, 2014 3:05 AM
To: Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

 

Abhishek,

 

In the case where a member crashes rpcs will not get unregistered with the system. I think we need to handle that scenario as well - in this scenario obviously the downed node will not be able to unregister the rpcs and not be able to handle a "role change" notification from the switch - which as per the OpenFlow Spec would not be sent to the slave controller anyway. One of the things which distinguishes a OF Master from a Slave controller is that slaves do not receive any asynchronous messages from the switch.

 

-Moiz

From: Abhishek Kumar (abhishk2)
Sent: Wednesday, June 04, 2014 2:08 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev@...; Basheeruddin Ahmed (syedbahm)
Subject: Re: Clustering impact on openflowplugin

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

 

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

 

-Abhishek

 

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:



The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

 

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

 

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

 

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

 

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

Please let me know if you believe this scheme would have problems.

 

Thanks,

-Moiz

 

Colin Dixon
 

Wait.

This should all be a performance optimization, right? That is if any node can access any MD-SAL data element, it's just it might be slow if it is remote. Similarly, any node can make any RPC call, it just might be slow if it's remote.

We might want to try to co-locate the data for a node on the same instance that has the connection (and thus handles the RPCs) for that node, but it shouldn't affect correctness.

Am I wrong?

--Colin


On Thu, Jun 5, 2014 at 12:47 PM, Moiz Raja (moraja) <moraja@...> wrote:
A cluster member.
From: Prasanna Huddar [prasanna.huddar@...]
Sent: Wednesday, June 04, 2014 10:29 PM
To: Moiz Raja (moraja); Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: RE: Clustering impact on openflowplugin

What does member indicate here?

-Prasanna

 

From: openflowplugin-dev-bounces@... [mailto:openflowplugin-dev-bounces@...] On Behalf Of Moiz Raja (moraja)
Sent: Thursday, June 05, 2014 3:05 AM
To: Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

 

Abhishek,

 

In the case where a member crashes rpcs will not get unregistered with the system. I think we need to handle that scenario as well - in this scenario obviously the downed node will not be able to unregister the rpcs and not be able to handle a "role change" notification from the switch - which as per the OpenFlow Spec would not be sent to the slave controller anyway. One of the things which distinguishes a OF Master from a Slave controller is that slaves do not receive any asynchronous messages from the switch.

 

-Moiz

From: Abhishek Kumar (abhishk2)
Sent: Wednesday, June 04, 2014 2:08 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev@...; Basheeruddin Ahmed (syedbahm)
Subject: Re: Clustering impact on openflowplugin

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

 

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

 

-Abhishek

 

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:



The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

 

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

 

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

 

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

 

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

Please let me know if you believe this scheme would have problems.

 

Thanks,

-Moiz

 


_______________________________________________
openflowjava-dev mailing list
openflowjava-dev@...
https://lists.opendaylight.org/mailman/listinfo/openflowjava-dev


Moiz Raja
 

Not wrong. It is a performance optimization only.

-Moiz

From: Colin Dixon [colin@...]
Sent: Thursday, June 05, 2014 11:12 AM
To: Moiz Raja (moraja)
Cc: Prasanna Huddar; Abhishek Kumar (abhishk2); openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowjava-dev] Clustering impact on openflowplugin

Wait.

This should all be a performance optimization, right? That is if any node can access any MD-SAL data element, it's just it might be slow if it is remote. Similarly, any node can make any RPC call, it just might be slow if it's remote.

We might want to try to co-locate the data for a node on the same instance that has the connection (and thus handles the RPCs) for that node, but it shouldn't affect correctness.

Am I wrong?

--Colin


On Thu, Jun 5, 2014 at 12:47 PM, Moiz Raja (moraja) <moraja@...> wrote:
A cluster member.
From: Prasanna Huddar [prasanna.huddar@...]
Sent: Wednesday, June 04, 2014 10:29 PM
To: Moiz Raja (moraja); Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: RE: Clustering impact on openflowplugin

What does member indicate here?

-Prasanna

 

From: openflowplugin-dev-bounces@... [mailto:openflowplugin-dev-bounces@...] On Behalf Of Moiz Raja (moraja)
Sent: Thursday, June 05, 2014 3:05 AM
To: Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

 

Abhishek,

 

In the case where a member crashes rpcs will not get unregistered with the system. I think we need to handle that scenario as well - in this scenario obviously the downed node will not be able to unregister the rpcs and not be able to handle a "role change" notification from the switch - which as per the OpenFlow Spec would not be sent to the slave controller anyway. One of the things which distinguishes a OF Master from a Slave controller is that slaves do not receive any asynchronous messages from the switch.

 

-Moiz

From: Abhishek Kumar (abhishk2)
Sent: Wednesday, June 04, 2014 2:08 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev@...; Basheeruddin Ahmed (syedbahm)
Subject: Re: Clustering impact on openflowplugin

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

 

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

 

-Abhishek

 

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:



The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

 

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

 

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

 

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

 

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

Please let me know if you believe this scheme would have problems.

 

Thanks,

-Moiz

 


_______________________________________________
openflowjava-dev mailing list
openflowjava-dev@...
https://lists.opendaylight.org/mailman/listinfo/openflowjava-dev


Colin Dixon
 

So, really, we'd like for there to be some way to figure out what controller instance is currently attached to a given device so that we could give it's data affinity to that device. Does the OpenFlow plugin expose the information about which switches are attached to which instances?

--Colin


On Thu, Jun 5, 2014 at 1:22 PM, Moiz Raja (moraja) <moraja@...> wrote:
Not wrong. It is a performance optimization only.

-Moiz
From: Colin Dixon [colin@...]
Sent: Thursday, June 05, 2014 11:12 AM
To: Moiz Raja (moraja)
Cc: Prasanna Huddar; Abhishek Kumar (abhishk2); openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowjava-dev] Clustering impact on openflowplugin

Wait.

This should all be a performance optimization, right? That is if any node can access any MD-SAL data element, it's just it might be slow if it is remote. Similarly, any node can make any RPC call, it just might be slow if it's remote.

We might want to try to co-locate the data for a node on the same instance that has the connection (and thus handles the RPCs) for that node, but it shouldn't affect correctness.

Am I wrong?

--Colin


On Thu, Jun 5, 2014 at 12:47 PM, Moiz Raja (moraja) <moraja@...> wrote:
A cluster member.
From: Prasanna Huddar [prasanna.huddar@...]
Sent: Wednesday, June 04, 2014 10:29 PM
To: Moiz Raja (moraja); Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: RE: Clustering impact on openflowplugin

What does member indicate here?

-Prasanna

 

From: openflowplugin-dev-bounces@... [mailto:openflowplugin-dev-bounces@...] On Behalf Of Moiz Raja (moraja)
Sent: Thursday, June 05, 2014 3:05 AM
To: Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

 

Abhishek,

 

In the case where a member crashes rpcs will not get unregistered with the system. I think we need to handle that scenario as well - in this scenario obviously the downed node will not be able to unregister the rpcs and not be able to handle a "role change" notification from the switch - which as per the OpenFlow Spec would not be sent to the slave controller anyway. One of the things which distinguishes a OF Master from a Slave controller is that slaves do not receive any asynchronous messages from the switch.

 

-Moiz

From: Abhishek Kumar (abhishk2)
Sent: Wednesday, June 04, 2014 2:08 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev@...; Basheeruddin Ahmed (syedbahm)
Subject: Re: Clustering impact on openflowplugin

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

 

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

 

-Abhishek

 

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:



The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

 

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

 

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

 

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

 

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

Please let me know if you believe this scheme would have problems.

 

Thanks,

-Moiz

 


_______________________________________________
openflowjava-dev mailing list
openflowjava-dev@...
https://lists.opendaylight.org/mailman/listinfo/openflowjava-dev



Abhijit Kumbhare
 

Moiz,

Couple of weeks ago Chris Price and I were discussing the implementation along the lines you are talking about. Master-slave - one of the controller cluster members is a master for a particular switch. The others indicate via set-async configuration that they are not interested in the async messages like packet-ins. Also the OpenFlow switch will likely be connecting to multiple controllers (from the cluster) - whichever controller gets connected first - will be the master for that switch. We assumed that there would be an arbiter which would decide which of the controller instance is a master. So we would need the OpenFlow plugin to let know the arbiter when handshake with a switch is completed. Then the arbiter can call this makeMaster method.

While this (master-slave - switches configured with multiple controllers IP addresses) is one possible implementation -  another possible implementation could be the switches actually have a single IP address for the controller (cluster) and connect to that IP address. I assume we are not thinking of that model for clustering - correct?

- Abhijit


On Wed, Jun 4, 2014 at 1:10 PM, Moiz Raja <moraja@...> wrote:
The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

Please let me know if you believe this scheme would have problems.

Thanks,
-Moiz

_______________________________________________
openflowjava-dev mailing list
openflowjava-dev@...
https://lists.opendaylight.org/mailman/listinfo/openflowjava-dev


Abhijit Kumbhare
 

Currently we don't have multi controller support in the OpenFlow plugin - so does not expose the information which switches are attached to which instance.


On Thu, Jun 5, 2014 at 11:58 AM, Colin Dixon <colin@...> wrote:

So, really, we'd like for there to be some way to figure out what controller instance is currently attached to a given device so that we could give it's data affinity to that device. Does the OpenFlow plugin expose the information about which switches are attached to which instances?

--Colin


On Thu, Jun 5, 2014 at 1:22 PM, Moiz Raja (moraja) <moraja@...> wrote:
Not wrong. It is a performance optimization only.

-Moiz
From: Colin Dixon [colin@...]
Sent: Thursday, June 05, 2014 11:12 AM
To: Moiz Raja (moraja)
Cc: Prasanna Huddar; Abhishek Kumar (abhishk2); openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowjava-dev] Clustering impact on openflowplugin

Wait.

This should all be a performance optimization, right? That is if any node can access any MD-SAL data element, it's just it might be slow if it is remote. Similarly, any node can make any RPC call, it just might be slow if it's remote.

We might want to try to co-locate the data for a node on the same instance that has the connection (and thus handles the RPCs) for that node, but it shouldn't affect correctness.

Am I wrong?

--Colin


On Thu, Jun 5, 2014 at 12:47 PM, Moiz Raja (moraja) <moraja@...> wrote:
A cluster member.
From: Prasanna Huddar [prasanna.huddar@...]
Sent: Wednesday, June 04, 2014 10:29 PM
To: Moiz Raja (moraja); Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: RE: Clustering impact on openflowplugin

What does member indicate here?

-Prasanna

 

From: openflowplugin-dev-bounces@... [mailto:openflowplugin-dev-bounces@...] On Behalf Of Moiz Raja (moraja)
Sent: Thursday, June 05, 2014 3:05 AM
To: Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

 

Abhishek,

 

In the case where a member crashes rpcs will not get unregistered with the system. I think we need to handle that scenario as well - in this scenario obviously the downed node will not be able to unregister the rpcs and not be able to handle a "role change" notification from the switch - which as per the OpenFlow Spec would not be sent to the slave controller anyway. One of the things which distinguishes a OF Master from a Slave controller is that slaves do not receive any asynchronous messages from the switch.

 

-Moiz

From: Abhishek Kumar (abhishk2)
Sent: Wednesday, June 04, 2014 2:08 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev@...; Basheeruddin Ahmed (syedbahm)
Subject: Re: Clustering impact on openflowplugin

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

 

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

 

-Abhishek

 

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:



The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

 

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

 

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

 

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

 

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

Please let me know if you believe this scheme would have problems.

 

Thanks,

-Moiz

 


_______________________________________________
openflowjava-dev mailing list
openflowjava-dev@...
https://lists.opendaylight.org/mailman/listinfo/openflowjava-dev




_______________________________________________
openflowjava-dev mailing list
openflowjava-dev@...
https://lists.opendaylight.org/mailman/listinfo/openflowjava-dev


Moiz Raja
 

Abhijit,

The option I was thinking of using was the one where we had an arbiter decide which controller would be master. How it decides could be based on various possible options like (a) is this controller colocated with inventory/topology (b) Does this controller have the capacity to handle another openflow connection and so on.

The other option that you proposed I wasn't thinking about but it is an interesting option. I am assuming you just want to use a VIP for the controller nodes and point all switches to that - am I right? In this case when a controller that was connected to a set of switches goes down and the switches reconnect to another controller is there anything different that the openflowplugin would need to do?

-Moiz

From: Abhijit Kumbhare [abhijitkoss@...]
Sent: Thursday, June 05, 2014 3:57 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev
Subject: Re: [openflowjava-dev] Clustering impact on openflowplugin

Moiz,

Couple of weeks ago Chris Price and I were discussing the implementation along the lines you are talking about. Master-slave - one of the controller cluster members is a master for a particular switch. The others indicate via set-async configuration that they are not interested in the async messages like packet-ins. Also the OpenFlow switch will likely be connecting to multiple controllers (from the cluster) - whichever controller gets connected first - will be the master for that switch. We assumed that there would be an arbiter which would decide which of the controller instance is a master. So we would need the OpenFlow plugin to let know the arbiter when handshake with a switch is completed. Then the arbiter can call this makeMaster method.

While this (master-slave - switches configured with multiple controllers IP addresses) is one possible implementation -  another possible implementation could be the switches actually have a single IP address for the controller (cluster) and connect to that IP address. I assume we are not thinking of that model for clustering - correct?

- Abhijit


On Wed, Jun 4, 2014 at 1:10 PM, Moiz Raja <moraja@...> wrote:
The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

Please let me know if you believe this scheme would have problems.

Thanks,
-Moiz

_______________________________________________
openflowjava-dev mailing list
openflowjava-dev@...
https://lists.opendaylight.org/mailman/listinfo/openflowjava-dev


Kamal Rameshan (kramesha) <kramesha@...>
 

Hi

For multi controller support, the open flow plugin would need an arbiter component. 


Please let know if any questions/concerns

-Kamal

From: "Moiz Raja (moraja)" <moraja@...>
Date: Thursday, June 5, 2014 10:47 AM
To: Prasanna Huddar <prasanna.huddar@...>, "Abhishek Kumar (abhishk2)" <abhishk2@...>
Cc: "openflowjava-dev@..." <openflowjava-dev@...>, "openflowplugin-dev@..." <openflowplugin-dev@...>
Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

A cluster member.

From: Prasanna Huddar [prasanna.huddar@...]
Sent: Wednesday, June 04, 2014 10:29 PM
To: Moiz Raja (moraja); Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: RE: Clustering impact on openflowplugin

What does member indicate here?

-Prasanna

 

From: openflowplugin-dev-bounces@... [mailto:openflowplugin-dev-bounces@...] On Behalf Of Moiz Raja (moraja)
Sent: Thursday, June 05, 2014 3:05 AM
To: Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

 

Abhishek,

 

In the case where a member crashes rpcs will not get unregistered with the system. I think we need to handle that scenario as well - in this scenario obviously the downed node will not be able to unregister the rpcs and not be able to handle a "role change" notification from the switch - which as per the OpenFlow Spec would not be sent to the slave controller anyway. One of the things which distinguishes a OF Master from a Slave controller is that slaves do not receive any asynchronous messages from the switch.

 

-Moiz

From: Abhishek Kumar (abhishk2)
Sent: Wednesday, June 04, 2014 2:08 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev@...; Basheeruddin Ahmed (syedbahm)
Subject: Re: Clustering impact on openflowplugin

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

 

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

 

-Abhishek

 

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:



The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

 

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

 

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

 

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

 

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

Please let me know if you believe this scheme would have problems.

 

Thanks,

-Moiz

 

Abhijit Kumbhare
 

Great. Why don't you talk about it a bit tomorrow?


On Mon, Jul 21, 2014 at 3:24 PM, Kamal Rameshan (kramesha) <kramesha@...> wrote:
Hi

For multi controller support, the open flow plugin would need an arbiter component. 


Please let know if any questions/concerns

-Kamal

From: "Moiz Raja (moraja)" <moraja@...>
Date: Thursday, June 5, 2014 10:47 AM
To: Prasanna Huddar <prasanna.huddar@...>, "Abhishek Kumar (abhishk2)" <abhishk2@...>
Cc: "openflowjava-dev@..." <openflowjava-dev@...>, "openflowplugin-dev@..." <openflowplugin-dev@...>

Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

A cluster member.
From: Prasanna Huddar [prasanna.huddar@...]
Sent: Wednesday, June 04, 2014 10:29 PM
To: Moiz Raja (moraja); Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: RE: Clustering impact on openflowplugin

What does member indicate here?

-Prasanna

 

From: openflowplugin-dev-bounces@... [mailto:openflowplugin-dev-bounces@...] On Behalf Of Moiz Raja (moraja)
Sent: Thursday, June 05, 2014 3:05 AM
To: Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

 

Abhishek,

 

In the case where a member crashes rpcs will not get unregistered with the system. I think we need to handle that scenario as well - in this scenario obviously the downed node will not be able to unregister the rpcs and not be able to handle a "role change" notification from the switch - which as per the OpenFlow Spec would not be sent to the slave controller anyway. One of the things which distinguishes a OF Master from a Slave controller is that slaves do not receive any asynchronous messages from the switch.

 

-Moiz

From: Abhishek Kumar (abhishk2)
Sent: Wednesday, June 04, 2014 2:08 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev@...; Basheeruddin Ahmed (syedbahm)
Subject: Re: Clustering impact on openflowplugin

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

 

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

 

-Abhishek

 

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:



The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

 

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

 

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

 

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

 

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

Please let me know if you believe this scheme would have problems.

 

Thanks,

-Moiz

 

_______________________________________________
openflowjava-dev mailing list
openflowjava-dev@...
https://lists.opendaylight.org/mailman/listinfo/openflowjava-dev


Kamal Rameshan (kramesha) <kramesha@...>
 

Yeah sure. Will present the high level design tomorrow.

-Kamal

From: Abhijit Kumbhare <abhijitkoss@...>
Date: Monday, July 21, 2014 3:54 PM
To: Cisco Employee <kramesha@...>
Cc: "openflowplugin-dev@..." <openflowplugin-dev@...>, "openflowjava-dev@..." <openflowjava-dev@...>
Subject: Re: [openflowjava-dev] Clustering impact on openflowplugin

Great. Why don't you talk about it a bit tomorrow?


On Mon, Jul 21, 2014 at 3:24 PM, Kamal Rameshan (kramesha) <kramesha@...> wrote:

Hi

For multi controller support, the open flow plugin would need an arbiter component. 


Please let know if any questions/concerns

-Kamal

From: "Moiz Raja (moraja)" <moraja@...>
Date: Thursday, June 5, 2014 10:47 AM
To: Prasanna Huddar <prasanna.huddar@...>, "Abhishek Kumar (abhishk2)" <abhishk2@...>
Cc: "openflowjava-dev@..." <openflowjava-dev@...>, "openflowplugin-dev@..." <openflowplugin-dev@...>

Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

A cluster member.
From: Prasanna Huddar [prasanna.huddar@...]
Sent: Wednesday, June 04, 2014 10:29 PM
To: Moiz Raja (moraja); Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: RE: Clustering impact on openflowplugin

What does member indicate here?

-Prasanna

 

From: openflowplugin-dev-bounces@... [mailto:openflowplugin-dev-bounces@...] On Behalf Of Moiz Raja (moraja)
Sent: Thursday, June 05, 2014 3:05 AM
To: Abhishek Kumar (abhishk2)
Cc: openflowjava-dev@...; openflowplugin-dev@...
Subject: Re: [openflowplugin-dev] Clustering impact on openflowplugin

 

Abhishek,

 

In the case where a member crashes rpcs will not get unregistered with the system. I think we need to handle that scenario as well - in this scenario obviously the downed node will not be able to unregister the rpcs and not be able to handle a "role change" notification from the switch - which as per the OpenFlow Spec would not be sent to the slave controller anyway. One of the things which distinguishes a OF Master from a Slave controller is that slaves do not receive any asynchronous messages from the switch.

 

-Moiz

From: Abhishek Kumar (abhishk2)
Sent: Wednesday, June 04, 2014 2:08 PM
To: Moiz Raja (moraja)
Cc: openflowplugin-dev@...; openflowjava-dev@...; Basheeruddin Ahmed (syedbahm)
Subject: Re: Clustering impact on openflowplugin

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

When a new node is discovered by the open flow plugin, it does routed rpc registration for this node with broker. When connection with new node is lost, the rpcs are unregistered for this node. OFPlugin folks - please correct me if I am wrong.

 

With above in place, when a controller is “made” master for a node, the OFPlugin on that controller can register for routed rpcs for this node. And similarly, the old master needs to unregister the rpcs for this node. Does openflow spec define any event that “old” master would get when its NOT master anymore?

 

-Abhishek

 

On Jun 4, 2014, at 1:10 PM, Moiz Raja <moraja@...> wrote:



The Clustering design in Helium is based on a Master-Slave model where all writes essentially go to the primary replica. In looking at the OF 1.3 Spec it appears that OF also supports the notion of Master-Slave controller and it seems that it would be the right mode of operation for us to use.

 

To properly support Master-Slave mode I believe the openflowplugin needs to do the following,

 

a. Support a global RPC method to make a specific controller the master (makeMaster). This method will be invoked by an external component. Since the clustering scheme is based on data shards we could come up with a simple rule that a member that has the inventory shard will also use the openflowplugin. When the external component notices that the primary shard for inventory has changed to the current cluster member then it will invoke the makeMaster RPC on the local openflowplugin.

 

b. The makeMaster implementation would need to send a message to the switch requesting that it’s role be set to master

 

c. The makeMaster implementation would also need to re-register all routed Rpc’s that were registered by the previous master. I haven’t thought through exactly how the new master would discover those Rpc’s - but one way would be enhance the Rpc registration API to include some meta information which could be used to query for those Rpc registrations. Another option could be enhance the RpcProvider to switch Rpc Registrations to the new provider.

 

Please let me know if you believe this scheme would have problems.

 

Thanks,

-Moiz

 


_______________________________________________
openflowjava-dev mailing list
openflowjava-dev@...
https://lists.opendaylight.org/mailman/listinfo/openflowjava-dev


1 - 18 of 18