[integration-dev] [opendaylight-dev] openflow performance regression
|
Luhrsen, Jamo <james.luhrsen@...>
I notified openflowplugin-dev about this yesterday. I was assuming this might have
toggle quoted message
Show quoted text
been the patch [1] that reduced the performance, as it was what looked like the patch that triggered the first test that saw the drop. But, that was not a commit from Robert, so now I’m not sure. also, please see the Lithium re-design performance plots [2], just to be able to compare. Thanks, JamO [1] https://git.opendaylight.org/gerrit/#/c/19638/ [2] https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-1node-cds-cbench-performance-lithium-redesign-only-master/plot/ On May 6, 2015, at 6:15 AM, Colin Dixon <colin@...<mailto:colin@...>> wrote:
I just heard that from Tony. Sorry for that. --Colin On Wed, May 6, 2015 at 9:11 AM, Mathieu Lemay <mlemay@...<mailto:mlemay@...>> wrote: Yes I think Robert sent out an e-mail on re-enabling notifications... Cheers Mathieu On Wed, May 6, 2015 at 9:06 AM, Colin Dixon <colin@...<mailto:colin@...>> wrote: https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-1node-cds-cbench-performance-only-master/plot/ Sometime around 19 hours ago it looks like our performance went off a cliff from ~100k flows per second to ~20k flows per second... --Colin _______________________________________________ dev mailing list dev@...<mailto:dev@...> https://lists.opendaylight.org/mailman/listinfo/dev -- [http://www.inocybe.com/wp-content/uploads/2014/09/default-login-image.png] Mathieu Lemay President & CEO Inocybe Technologies 1-888-445-7505<tel:1-888-445-7505> www.inocybe.com<http://www.inocybe.com/> _______________________________________________ integration-dev mailing list integration-dev@...<mailto:integration-dev@...> https://lists.opendaylight.org/mailman/listinfo/integration-dev |
|
|
|
Abhijit Kumbhare
Let's discuss this in the OpenFlow plugin IRC bug scrub meeting at 9 am Pacific today. On Wed, May 6, 2015 at 7:07 AM, Luhrsen, Jamo <james.luhrsen@...> wrote: I notified openflowplugin-dev about this yesterday. I was assuming this might have |
|
|
|
Luis Gomez <ecelgp@...>
Hi Jamo. The patch that reduced the performance is: You could not see this clearly in the CI because projects that have branched to stable/lithium like controller or yangtools are not triggering any test in CI. BR/Luis
|
|
|
|
Tony Tkacik
The patch https://git.opendaylight.org/gerrit/#/c/19258/ is not even in the code-path of throughput testing, since change was to clustered datastore And for throughput testing suite is using dropallpacketsrpc on Which is based on RPCs and Notifications only.
Tony
From: openflowjava-dev-bounces@... [mailto:openflowjava-dev-bounces@...]
On Behalf Of Luis Gomez
Sent: Wednesday, May 06, 2015 5:01 PM To: Luhrsen, Jamo Cc: openflowplugin-dev@...; integration-dev@...; openflowjava-dev@...; Mathieu Lemay Subject: Re: [openflowjava-dev] [integration-dev] [opendaylight-dev] openflow performance regression
Hi Jamo. The patch that reduced the performance is:
You could not see this clearly in the CI because projects that have branched to stable/lithium like controller or yangtools are not triggering any test in CI.
BR/Luis
|
|
|
|
Luis Gomez <ecelgp@...>
OK, you are right Tony, looking at the exact patches that went in the distribution that reduced performance: openflowplugin: controller: It seems the openflowplugin patch is more candidate than the other 2 from controller. BR/Luis
|
|
|
|
Abhijit Kumbhare
Luis, In today's OF Plugin meeting Tony provided the info on this & the fix: [09:08:50] <ttkacik1> whole issue is in droptest itself [09:09:40] <abhijitkumbhare> Oh - but have there been any changes to droptest? [09:09:54] <ttkacik1> droptest was using MD-SAL threads even if it is explicitly docuemnted in javadocs for MD-SAL Notifcation APIs to do not any work which may take time
[09:10:48] <abhijitkumbhare> so someone needs to change drop test [09:10:53] <ttkacik1> patch which dropped performance was announced to mailing lists before: https://lists.opendaylight.org/pipermail/openflowplugin-dev/2015-May/003022.html [09:11:08] <ttkacik1> even points to patch [09:11:30] <abhijitkumbhare> OK [09:11:54] <ttkacik1> which changed bit behavior of notif. broker impl but not API contractz [09:12:11] <ttkacik1> which showed that droptest was stealing MD-SAL threads to do its work [09:12:19] <ttkacik1> that is why perf. dropped [09:12:40] <ttkacik1> now notification broker makes sure none of the listener is invoked from multiple threads at same time [09:12:51] <abhijitkumbhare> so someone needs to change drop test to work better with the lithium notification broker [09:13:10] <abhijitkumbhare> right? [09:13:29] <ttkacik1> and that was implementation specific behaviour on which droptest relies [09:13:38] <ttkacik1> I pushed initial fix to droptest as of now [09:14:12] <abhijitkumbhare> OK - good [09:14:37] <jamoluhrsen> #info https://git.opendaylight.org/gerrit/#/c/19717/ is Tony's drop-test patch [09:14:39] <ttkacik1> https://git.opendaylight.org/gerrit/#/c/19717/ [09:14:46] <michal_rehak> testing the patch right now [09:15:21] <abhijitkumbhare> OK [09:15:22] <michal_rehak> looks like it improves throughput in He-codebase of ofPlugin 3 times [09:16:03] <michal_rehak> merged Thanks, Abhijit On Wed, May 6, 2015 at 9:20 AM, Luis Gomez <ecelgp@...> wrote:
|
|
|