[net-virt-platform-dev] Flow handling in multiple flow tables -- ignore my previous mail


Rob Sherwood <rob.sherwood@...>
 

Hi Tech.kals,

Thanks for the interest in OF-DPA.  Can I ask: are you using the indigo ofagent code that ships with OFDPA?  If so, it maps every flow to an internal unique flow entry ID and all of the internal operations are based off that flow entry ID to avoid problems just like this.  To your point, yes, as with any multi-table system, the controller is responsible for keeping track of which flows belong in which tables as well as their interdependency.

Fwiw, I've started adding some OF-DPA tests to OFTest (https://github.com/floodlight/oftest/pull/189) which has been useful to others in understanding which flows to put in to which tables.  I'm right now at an interop event testing with ODL, so let me know if you find any issues.

- Rob
.

On Mon, Aug 3, 2015 at 11:40 PM, tech_kals Kals <tech.kals@...> wrote:
Hi experts,

  Please ignore my previous mail. Sent it without completion.

am trying to integrate Broadcom's OFDPA (OpenFlow Data Path Abstraction) into my data-plane. As you all know, OFDPA supports multiple flow tables for programming multiple flow entries.

when multiple flow tables are supported in the data plane hardware, we might face some issues and one shown below.

1) Flow deletion:
   Assume, Ingress flow table and VLAN flow table supported in the hardware.
   So, when you get a flow entry as shown below, using their match criteria the entry would be added into both the tables.

---------------------------------------------------------------------------------
Flow entry                     |             Match criteria           |
---------------------------------------------------------------------------------
 Entry 1                        |          inport : eth0,
                                    |          vlan  : 10    
---------------------------------------------------------------------------------

 This is fine. When you want to delete the same match criteria would be used to uniquely find a flow entry {eth0 is key for ingress flow table & vlan10 is the key for vlan table) in each of the flow table.

Now, if you get 2 flow entry as shown below, how the hardware will behave.

---------------------------------------------------------------------------------
Flow entry                     |             Match criteria           |
---------------------------------------------------------------------------------
 Entry 1                         |          inport : eth0,
                                     |          vlan  : 10    
---------------------------------------------------------------------------------
 Entry 2                         |          inport : eth0,
                                    |          vlan  : 20    
---------------------------------------------------------------------------------

If you look at the flow tables in hardware, it looks as shown below.

--------------------------------------------------------------------------------------------------------------
Flow entry                     |            ingress table            |           vlan table 
-------------------------------------------------------------------------------------------------------------
 Entry 1                         |          inport : eth0               |            vlan 10
                                                                                |            vlan 20                                      
--------------------------------------------------------------------------------------------------------------

Here, the problem starts....

Ingress table entry is common for both vlan entries. So, if you want to delete the 2nd flow entry in VLAN table then it should be deleted from Ingress table as well.
If you delete the flow entry from Ingress flow table then both flow entries in VLAN table would get deleted. So, how this can be handled?

How controller will handle this case? I assume, the controller will send flow add and flow delete message to each flow table separately. If so, will controller maintains some data which would tell like the flow entry (in our case ingress table flow entry ) is being referred in some other table? Just curious to know about it.


Thanks,
tech.kals

_______________________________________________
net-virt-platform-dev mailing list
net-virt-platform-dev@...
https://lists.opendaylight.org/mailman/listinfo/net-virt-platform-dev



kals
 

Hi Rob,
 
   Thanks for your response.

 1) Am not using Indigo agent. we have our own OF Agent and using OVS ofproto to interact with controller & have written our own ofproto-dpif to program our Openflow Switch.

2) As TTP support is *not* currently there on OVS and on any controller, our OF Agent is not able to convey the hardware capability (including OFDPA supported Openflow tables capability) to Controller through OVS. So, am planning to write a glue layer in our code to distribute the incoming elephant flow to all the required flow tables. If any Openflow agent & contoller supports TTP then it would be so helpful for me. Could you please let me know if any Openflow agent and Controller supports TTP ? 

3) Are you sure, Indigo supports TTP?

Thanks,
tech.kals




On Wed, Aug 5, 2015 at 12:05 AM, Rob Sherwood <rob.sherwood@...> wrote:
Hi Tech.kals,

Thanks for the interest in OF-DPA.  Can I ask: are you using the indigo ofagent code that ships with OFDPA?  If so, it maps every flow to an internal unique flow entry ID and all of the internal operations are based off that flow entry ID to avoid problems just like this.  To your point, yes, as with any multi-table system, the controller is responsible for keeping track of which flows belong in which tables as well as their interdependency.

Fwiw, I've started adding some OF-DPA tests to OFTest (https://github.com/floodlight/oftest/pull/189) which has been useful to others in understanding which flows to put in to which tables.  I'm right now at an interop event testing with ODL, so let me know if you find any issues.

- Rob
.

On Mon, Aug 3, 2015 at 11:40 PM, tech_kals Kals <tech.kals@...> wrote:
Hi experts,

  Please ignore my previous mail. Sent it without completion.

am trying to integrate Broadcom's OFDPA (OpenFlow Data Path Abstraction) into my data-plane. As you all know, OFDPA supports multiple flow tables for programming multiple flow entries.

when multiple flow tables are supported in the data plane hardware, we might face some issues and one shown below.

1) Flow deletion:
   Assume, Ingress flow table and VLAN flow table supported in the hardware.
   So, when you get a flow entry as shown below, using their match criteria the entry would be added into both the tables.

---------------------------------------------------------------------------------
Flow entry                     |             Match criteria           |
---------------------------------------------------------------------------------
 Entry 1                        |          inport : eth0,
                                    |          vlan  : 10    
---------------------------------------------------------------------------------

 This is fine. When you want to delete the same match criteria would be used to uniquely find a flow entry {eth0 is key for ingress flow table & vlan10 is the key for vlan table) in each of the flow table.

Now, if you get 2 flow entry as shown below, how the hardware will behave.

---------------------------------------------------------------------------------
Flow entry                     |             Match criteria           |
---------------------------------------------------------------------------------
 Entry 1                         |          inport : eth0,
                                     |          vlan  : 10    
---------------------------------------------------------------------------------
 Entry 2                         |          inport : eth0,
                                    |          vlan  : 20    
---------------------------------------------------------------------------------

If you look at the flow tables in hardware, it looks as shown below.

--------------------------------------------------------------------------------------------------------------
Flow entry                     |            ingress table            |           vlan table 
-------------------------------------------------------------------------------------------------------------
 Entry 1                         |          inport : eth0               |            vlan 10
                                                                                |            vlan 20                                      
--------------------------------------------------------------------------------------------------------------

Here, the problem starts....

Ingress table entry is common for both vlan entries. So, if you want to delete the 2nd flow entry in VLAN table then it should be deleted from Ingress table as well.
If you delete the flow entry from Ingress flow table then both flow entries in VLAN table would get deleted. So, how this can be handled?

How controller will handle this case? I assume, the controller will send flow add and flow delete message to each flow table separately. If so, will controller maintains some data which would tell like the flow entry (in our case ingress table flow entry ) is being referred in some other table? Just curious to know about it.


Thanks,
tech.kals

_______________________________________________
net-virt-platform-dev mailing list
net-virt-platform-dev@...
https://lists.opendaylight.org/mailman/listinfo/net-virt-platform-dev




Rob Sherwood <rob.sherwood@...>
 

[sorry for the slow replies; I'm traveling]

re (1): The OF-DPA daemon is responsible for mapping OF-level resources to hardware level resources and if there is an overlap in those resources (as you've identified), it's supposed to do the reference counting, etc. to make sure that those dependencies are properly managed.  Have you actually found a case where that's not true or are you just wondering hypothetically?  It's entirely possible that you've found a bug in which case it would be useful to try to make it easy to reproduce (e.g., via an OFTest).

re (2) and (3): "TTP support" is much simpler than I think you understand.  TTP's are just a read only description of the pipeline.  In theory a pipeline could support multiple TTP's and the controller could then pick one, thus causing the chip to reprogram itself, but OF-DPA-based openflow agents only support a single pipeline so there's only one TTP (https://github.com/Broadcom-Switch/of-dpa/blob/master/OF-DPA-2.0/ofdpa-v2.0.0.2.ttp.json) and so the agent trivially "supports" it.  Indigo does not support OFConfig yet, so the 'official' mechanism to set the TTPs does not exist, but since there is only one TTP anyway, it's not clear what value there would be in implementing that knob.


Let me know if that helps and thanks,

- Rob
.



On Wed, Aug 5, 2015 at 12:37 PM, tech_kals Kals <tech.kals@...> wrote:
Hi Rob,
 
   Thanks for your response.

 1) Am not using Indigo agent. we have our own OF Agent and using OVS ofproto to interact with controller & have written our own ofproto-dpif to program our Openflow Switch.

2) As TTP support is *not* currently there on OVS and on any controller, our OF Agent is not able to convey the hardware capability (including OFDPA supported Openflow tables capability) to Controller through OVS. So, am planning to write a glue layer in our code to distribute the incoming elephant flow to all the required flow tables. If any Openflow agent & contoller supports TTP then it would be so helpful for me. Could you please let me know if any Openflow agent and Controller supports TTP ? 

3) Are you sure, Indigo supports TTP?

Thanks,
tech.kals




On Wed, Aug 5, 2015 at 12:05 AM, Rob Sherwood <rob.sherwood@...> wrote:
Hi Tech.kals,

Thanks for the interest in OF-DPA.  Can I ask: are you using the indigo ofagent code that ships with OFDPA?  If so, it maps every flow to an internal unique flow entry ID and all of the internal operations are based off that flow entry ID to avoid problems just like this.  To your point, yes, as with any multi-table system, the controller is responsible for keeping track of which flows belong in which tables as well as their interdependency.

Fwiw, I've started adding some OF-DPA tests to OFTest (https://github.com/floodlight/oftest/pull/189) which has been useful to others in understanding which flows to put in to which tables.  I'm right now at an interop event testing with ODL, so let me know if you find any issues.

- Rob
.

On Mon, Aug 3, 2015 at 11:40 PM, tech_kals Kals <tech.kals@...> wrote:
Hi experts,

  Please ignore my previous mail. Sent it without completion.

am trying to integrate Broadcom's OFDPA (OpenFlow Data Path Abstraction) into my data-plane. As you all know, OFDPA supports multiple flow tables for programming multiple flow entries.

when multiple flow tables are supported in the data plane hardware, we might face some issues and one shown below.

1) Flow deletion:
   Assume, Ingress flow table and VLAN flow table supported in the hardware.
   So, when you get a flow entry as shown below, using their match criteria the entry would be added into both the tables.

---------------------------------------------------------------------------------
Flow entry                     |             Match criteria           |
---------------------------------------------------------------------------------
 Entry 1                        |          inport : eth0,
                                    |          vlan  : 10    
---------------------------------------------------------------------------------

 This is fine. When you want to delete the same match criteria would be used to uniquely find a flow entry {eth0 is key for ingress flow table & vlan10 is the key for vlan table) in each of the flow table.

Now, if you get 2 flow entry as shown below, how the hardware will behave.

---------------------------------------------------------------------------------
Flow entry                     |             Match criteria           |
---------------------------------------------------------------------------------
 Entry 1                         |          inport : eth0,
                                     |          vlan  : 10    
---------------------------------------------------------------------------------
 Entry 2                         |          inport : eth0,
                                    |          vlan  : 20    
---------------------------------------------------------------------------------

If you look at the flow tables in hardware, it looks as shown below.

--------------------------------------------------------------------------------------------------------------
Flow entry                     |            ingress table            |           vlan table 
-------------------------------------------------------------------------------------------------------------
 Entry 1                         |          inport : eth0               |            vlan 10
                                                                                |            vlan 20                                      
--------------------------------------------------------------------------------------------------------------

Here, the problem starts....

Ingress table entry is common for both vlan entries. So, if you want to delete the 2nd flow entry in VLAN table then it should be deleted from Ingress table as well.
If you delete the flow entry from Ingress flow table then both flow entries in VLAN table would get deleted. So, how this can be handled?

How controller will handle this case? I assume, the controller will send flow add and flow delete message to each flow table separately. If so, will controller maintains some data which would tell like the flow entry (in our case ingress table flow entry ) is being referred in some other table? Just curious to know about it.


Thanks,
tech.kals

_______________________________________________
net-virt-platform-dev mailing list
net-virt-platform-dev@...
https://lists.opendaylight.org/mailman/listinfo/net-virt-platform-dev