Barrier Message Timeout - 500ms - Rational behind the value
|
Muthukrishnan T
Dear Team ,
I have been working in Lithium and Boron Stable Version .
Lets take Boron , In openflowplugin.cfg default timer for barrier message is 500ms is there any rational behind the same ? why not 100ms ?
If I push via SAL-FLOW RPC , total time taken for flow push = Time taken for flow push + barrier delay = ~ 30 ms + 520 ms (barrier ).
If I change the value from 500ms to 100 ms I am getting drastic performance improvement in flow push processing . Can anyone tell why we have gone for 500ms as default timer value ? How we have arrived this golden value of 500ms ?
Reference.
File : openflowplugin.cfg "#Timeout interval in milliseconds between each barrier message. #Default value is set to 500 milliseconds
barrier-interval-timeout-limit=500"
Regards
Muthukrishnan
9952012433
|
|
|
|
Michal Rehak -X (mirehak - PANTHEON TECHNOLOGIES@Cisco) <mirehak@...>
Hi Muthukrishnan, there is no golden standard behind this value. Setting it to 100ms or even lower value will result into low latency system but this also decreases throughput of system.
If you have few switches and little no big requirement on flows per second throughput then 100ms might suit better to your setup. On the other hand if there are hundreds of devices and throughput of thousands of flows per second required then 500ms or even higher will improve overall performance.
Also there is a complementary parameter - amount of messages in outbound queue without barrier inside. The more messages kept in queue the higher throughput you shall get. But there is memory cost and also by extending size of those queues (per device) there comes overload
risk of md-sal notification queue.
True - this value might be configurable. But that would solve the situation only for static requirements.
Regards, Michal
From: openflowjava-dev-bounces@... <openflowjava-dev-bounces@...> on behalf of Muthukrishnan Thangasamy <muthukrishnan.t@...>
Sent: Wednesday, February 15, 2017 08:13 To: openflowplugin-dev@... Cc: openflowjava-dev@... Subject: [openflowjava-dev] Barrier Message Timeout - 500ms - Rational behind the value
Dear Team ,
I have been working in Lithium and Boron Stable Version .
Lets take Boron , In openflowplugin.cfg default timer for barrier message is 500ms is there any rational behind the same ? why not 100ms ?
If I push via SAL-FLOW RPC , total time taken for flow push = Time taken for flow push + barrier delay = ~ 30 ms + 520 ms (barrier ).
If I change the value from 500ms to 100 ms I am getting drastic performance improvement in flow push processing . Can anyone tell why we have gone for 500ms as default timer value ? How we have arrived this golden value of 500ms ?
Reference.
File : openflowplugin.cfg "#Timeout interval in milliseconds between each barrier message. #Default value is set to 500 milliseconds
barrier-interval-timeout-limit=500"
Regards
Muthukrishnan
9952012433
|
|
|
|
Muthukrishnan T
Hi Michal ,
Thank you for your response , I agree with you , I need some clarity on your response
Also there is a complementary parameter - amount of messages in outbound queue without barrier inside. The more messages kept in queue the higher throughput you shall get.
Question : My understanding is that you are referring to Maximum outbound queue depth which is 25600 by default , here if we are sending the flow via SAL-RPC and barrier timer is 500 ms how to achieve flow throughput of 1000 flows/sec ( total number of non-barrier message kept in queue as 1000) , since RPC call form northbound takes ( 500ms + 20 ms ) for writing single flow (assuming writing each flow sequentially in loop), throughput is less even you are sending 1000 flows/sec. If am sending via RPC how to keep 1000 openflow messages in queue without barrier ? is there any other parameter which is controlling the same ?
Regards Muthukrishnan 9952012433
From: Michal Rehak -X (mirehak - PANTHEON TECHNOLOGIES at Cisco) <mirehak@...>
Sent: Thursday, February 16, 2017 2:21 PM To: Muthukrishnan Thangasamy; openflowplugin-dev@... Cc: openflowjava-dev@... Subject: Re: [openflowjava-dev] Barrier Message Timeout - 500ms - Rational behind the value Hi Muthukrishnan, there is no golden standard behind this value. Setting it to 100ms or even lower value will result into low latency system but this also decreases throughput of system.
If you have few switches and little no big requirement on flows per second throughput then 100ms might suit better to your setup. On the other hand if there are hundreds of devices and throughput of thousands of flows per second required then 500ms or even higher will improve overall performance.
Also there is a complementary parameter - amount of messages in outbound queue without barrier inside. The more messages kept in queue the higher throughput you shall get. But there is memory cost and also by extending size of those queues (per device) there comes overload
risk of md-sal notification queue.
True - this value might be configurable. But that would solve the situation only for static requirements.
Regards, Michal
From: openflowjava-dev-bounces@... <openflowjava-dev-bounces@...> on behalf of Muthukrishnan Thangasamy <muthukrishnan.t@...>
Sent: Wednesday, February 15, 2017 08:13 To: openflowplugin-dev@... Cc: openflowjava-dev@... Subject: [openflowjava-dev] Barrier Message Timeout - 500ms - Rational behind the value
Dear Team ,
I have been working in Lithium and Boron Stable Version .
Lets take Boron , In openflowplugin.cfg default timer for barrier message is 500ms is there any rational behind the same ? why not 100ms ?
If I push via SAL-FLOW RPC , total time taken for flow push = Time taken for flow push + barrier delay = ~ 30 ms + 520 ms (barrier ).
If I change the value from 500ms to 100 ms I am getting drastic performance improvement in flow push processing . Can anyone tell why we have gone for 500ms as default timer value ? How we have arrived this golden value of 500ms ?
Reference.
File : openflowplugin.cfg "#Timeout interval in milliseconds between each barrier message. #Default value is set to 500 milliseconds
barrier-interval-timeout-limit=500"
Regards
Muthukrishnan
9952012433
|
|
|
|
Michal Rehak -X (mirehak - PANTHEON TECHNOLOGIES@Cisco) <mirehak@...>
Hi Muthukrishnan,
right - I was referring to 'Maximum outbound queue depth'. 25600 is not maximum amount of messages in outbound queue - it is just a threshold value and when crossed then automatically a barrier message is added to queue. But the weird thing about this is that the limit for barrier in queue (25.6k) should be actually smaller than rpc limit (20k). So if you are observing one switch throughput then I believe that by increasing rpc-limit to 30k or decreasing the barrier in queue limit to 15k will improve your performance.
See: openFlowPlugin/openflowplugin-api/src/main/yang/openflow-provider-config.yang openFlowPlugin/openflowplugin-blueprint-config/src/main/resources/initial/openflowplugin.cfg rpc-requests-quota
Regards, Michal
From: Muthukrishnan Thangasamy <muthukrishnan.t@...>
Sent: Wednesday, February 22, 2017 08:35 To: Michal Rehak -X (mirehak - PANTHEON TECHNOLOGIES at Cisco); openflowplugin-dev@... Cc: openflowjava-dev@... Subject: Re: [openflowjava-dev] Barrier Message Timeout - 500ms - Rational behind the value Hi Michal ,
Thank you for your response , I agree with you , I need some clarity on your response
Also there is a complementary parameter - amount of messages in outbound queue without barrier inside. The more messages kept in queue the higher throughput you shall get.
Question : My understanding is that you are referring to Maximum outbound queue depth which is 25600 by default , here if we are sending the flow via SAL-RPC and barrier timer is 500 ms how to achieve flow throughput of 1000 flows/sec ( total number of non-barrier message kept in queue as 1000) , since RPC call form northbound takes ( 500ms + 20 ms ) for writing single flow (assuming writing each flow sequentially in loop), throughput is less even you are sending 1000 flows/sec. If am sending via RPC how to keep 1000 openflow messages in queue without barrier ? is there any other parameter which is controlling the same ?
Regards Muthukrishnan 9952012433
From: Michal Rehak -X (mirehak - PANTHEON TECHNOLOGIES at Cisco) <mirehak@...>
Sent: Thursday, February 16, 2017 2:21 PM To: Muthukrishnan Thangasamy; openflowplugin-dev@... Cc: openflowjava-dev@... Subject: Re: [openflowjava-dev] Barrier Message Timeout - 500ms - Rational behind the value Hi Muthukrishnan, there is no golden standard behind this value. Setting it to 100ms or even lower value will result into low latency system but this also decreases throughput of system.
If you have few switches and little no big requirement on flows per second throughput then 100ms might suit better to your setup. On the other hand if there are hundreds of devices and throughput of thousands of flows per second required then 500ms or even higher will improve overall performance.
Also there is a complementary parameter - amount of messages in outbound queue without barrier inside. The more messages kept in queue the higher throughput you shall get. But there is memory cost and also by extending size of those queues (per device) there comes overload
risk of md-sal notification queue.
True - this value might be configurable. But that would solve the situation only for static requirements.
Regards, Michal
From: openflowjava-dev-bounces@... <openflowjava-dev-bounces@...> on behalf of Muthukrishnan Thangasamy <muthukrishnan.t@...>
Sent: Wednesday, February 15, 2017 08:13 To: openflowplugin-dev@... Cc: openflowjava-dev@... Subject: [openflowjava-dev] Barrier Message Timeout - 500ms - Rational behind the value
Dear Team ,
I have been working in Lithium and Boron Stable Version .
Lets take Boron , In openflowplugin.cfg default timer for barrier message is 500ms is there any rational behind the same ? why not 100ms ?
If I push via SAL-FLOW RPC , total time taken for flow push = Time taken for flow push + barrier delay = ~ 30 ms + 520 ms (barrier ).
If I change the value from 500ms to 100 ms I am getting drastic performance improvement in flow push processing . Can anyone tell why we have gone for 500ms as default timer value ? How we have arrived this golden value of 500ms ?
Reference.
File : openflowplugin.cfg "#Timeout interval in milliseconds between each barrier message. #Default value is set to 500 milliseconds
barrier-interval-timeout-limit=500"
Regards
Muthukrishnan
9952012433
|
|
|