Re: OPENFLOWJAVA System Test Waiver
Jamo Luhrsen <jluhrsen@...>
This was approved in last weeks integration meeting [0] and the wiki [1] has
toggle quoted message
Show quoted text
been updated. Thanks, JamO [0] https://meetings.opendaylight.org/opendaylight-integration/2016/integration/opendaylight-integration-integration.2016-05-19-16.01.html [1] https://wiki.opendaylight.org/view/Simultaneous_Release/Boron/Waiver/System_Test
On 05/18/2016 11:31 AM, An Ho wrote:
As indicated in their M2 Status Email [1], the OPENFLOWJAVA team would like to request a system test waiver for Boron [2]. The reason behind the |
|
Re: [opendaylight-dev] [OpenDaylight Discuss] TWS on reorganizing the OpenFlow projects
Colin Dixon
After thinking about this for a bit, I think we have two real current problems: 1.) It's confusing for people to figure out what projects are related to OpenFlow and how.On Fri, May 13, 2016 at 3:22 AM, Anil Vishnoi <vishnoianil@...> wrote:
|
|
OPENFLOWJAVA System Test Waiver
an.ho@huawei.com
As indicated in their M2 Status Email [1], the OPENFLOWJAVA team would like to request a system test waiver for Boron [2]. The reason behind the request is that openflowjava's functionality is tested by openflowplugin project which covers probably all OVS (virtual switch) supported messages / scenarios.
Best Regards, An Ho
[1] https://lists.opendaylight.org/pipermail/release/2016-April/006309.html
|
|
Re: New message count for Openflow Java in ODL master (Boron)
Chinmay Kumar Mohanta <chinmay.kumar.mohanta@...>
Hi Michal,
Thanks for your reply. Yes, I will do the changes as you have indicated.
Regards, Chinmay
From: Michal Polkoráb [mailto:michal.polkorab@...]
Hi Chinmay,
in that case please go ahead with your changes. You can also remove the two positive counters mentioned before to further improve the performance (we will still have the send messages counters from openflowplugin). Please push one separate commit - where you remove these counters, so that we can track it easily in case the counters are needed in the future.
Regards, Michal From: Chinmay Kumar Mohanta <chinmay.kumar.mohanta@...>
Hi Michal,
From our ongoing effort to debug some of the performance related issues we find that it greatly simplifies the debugging process if error cases (for example like message drop) are reported explicitly rather than implicitly (as a derivation). Explicit error reporting provides *** sensible & precise contextual*** information which will expedite the process of finding the root cause of the issue.
In line with this principle we are still strongly considering to implement this new counter.
Pleases comment?
Thanks and Regards, Chinmay Mohanta
From: Michal Polkoráb [mailto:michal.polkorab@...]
Hi Chinmay,
your point makes sense. However, we should thoroughly consider removing these counters. These were introduced in the past to check if / where there are messages being lost between openflowplugin & openflowjava + to verify the backpressure mechanism. If we similar issue appears again, these counters might be handy.
Michal From: Chinmay.Mohanta <mohanta.chinmay@...>
Hi Michal,
Thanks for your reply.
Indeed the formula indicated by you gives the expected value !! So it obviates the implementation of a new counter.
However following new thoughts emerge.
Do we really need the counts DS_ENTERED_OFJAVA and DS_ENCODE_SUCCESS ?
Maintaining these two counts (for every message) is costly in terms of performance (due to need for synchronization).
I think we should have counts only for the error scenarios resulting least influence on the performance.
BTW, my goal is to simplify the debugging process to find out whenever/wherever there is a message drop due to resource contention. Message drop count will provide an essential input to rationalize the provisioning of resources.
Thanks and Regards, Chinmay @ ericsson
On Wed, May 11, 2016 at 5:22 PM, Michal Polkoráb <michal.polkorab@...> wrote:
Hello Chinmay,
is it really needed to add such counter ? One can get this number by DS_ENTERED_OFJAVA - (DS_ENCODE_SUCCESS + DS_ENCODE_FAIL) if only OutboundQueue is used. This counter looses its meaning when OutboundQueueManager is used as the manager enqueues all messages and the message count is controlled byt rpcQuota in openflowplugin.
Regards, Michal
From: Chinmay.Mohanta <mohanta.chinmay@...>
Hi all,
To supplement present testing and diagnosing scenarios using message counts, I am proposing following new count.
Count name : DS_DROPPED_PACKET_QFULL Purpose : To count messages dropped because the queue present in ChannelOutboundQueue is full. This count will be helpful to mark the threshold downstream message processing capability of Openflow Java. Downsizing/upsizing resources surrounding this counter will decrease/increase downstream message processing capability.
Implementation implications:
This new count will be implemented using existing counter module StatisticsCounters.
Please share your comments (if any ).
Thanks and Regards, Chinmay @ ericsson MichalPolkoráb Software Engineer
MichalPolkoráb Software Engineer
MichalPolkoráb Software Engineer
|
|
Re: New message count for Openflow Java in ODL master (Boron)
Michal Polkoráb <michal.polkorab@...>
Hi Chinmay,
in that case please go ahead with your changes. You can also remove the two positive counters mentioned before to further improve the performance (we will still have the send messages counters from openflowplugin). Please push one separate commit - where
you remove these counters, so that we can track it easily in case the counters are needed in the future.
Regards, Michal From: Chinmay Kumar Mohanta <chinmay.kumar.mohanta@...>
Sent: 17 May 2016 08:10 To: Michal Polkoráb; Chinmay.Mohanta Cc: openflowjava-dev@...; Shuva Jyoti Kar; Ashutosh Bisht; Muthukumaran K Subject: RE: [openflowjava-dev] New message count for Openflow Java in ODL master (Boron) Hi Michal,
From our ongoing effort to debug some of the performance related issues we find that it greatly simplifies the debugging process if error cases (for example like message drop) are reported explicitly rather than implicitly (as a derivation). Explicit error reporting provides *** sensible & precise contextual*** information which will expedite the process of finding the root cause of the issue.
In line with this principle we are still strongly considering to implement this new counter.
Pleases comment?
Thanks and Regards, Chinmay Mohanta
From: Michal Polkoráb [mailto:michal.polkorab@...]
Hi Chinmay,
your point makes sense. However, we should thoroughly consider removing these counters. These were introduced in the past to check if / where there are messages being lost between openflowplugin & openflowjava + to verify the backpressure mechanism. If we similar issue appears again, these counters might be handy.
Michal From: Chinmay.Mohanta <mohanta.chinmay@...>
Hi Michal,
Thanks for your reply.
Indeed the formula indicated by you gives the expected value !! So it obviates the implementation of a new counter.
However following new thoughts emerge.
Do we really need the counts DS_ENTERED_OFJAVA and DS_ENCODE_SUCCESS ?
Maintaining these two counts (for every message) is costly in terms of performance (due to need for synchronization).
I think we should have counts only for the error scenarios resulting least influence on the performance.
BTW, my goal is to simplify the debugging process to find out whenever/wherever there is a message drop due to resource contention. Message drop count will provide an essential input to rationalize the provisioning of resources.
Thanks and Regards, Chinmay @ ericsson
On Wed, May 11, 2016 at 5:22 PM, Michal Polkoráb <michal.polkorab@...> wrote:
Hello Chinmay,
is it really needed to add such counter ? One can get this number by DS_ENTERED_OFJAVA - (DS_ENCODE_SUCCESS + DS_ENCODE_FAIL) if only OutboundQueue is used. This counter looses its meaning when OutboundQueueManager is used as the manager enqueues all messages and the message count is controlled byt rpcQuota in openflowplugin.
Regards, Michal
From: Chinmay.Mohanta <mohanta.chinmay@...>
Hi all,
To supplement present testing and diagnosing scenarios using message counts, I am proposing following new count.
Count name : DS_DROPPED_PACKET_QFULL Purpose : To count messages dropped because the queue present in ChannelOutboundQueue is full. This count will be helpful to mark the threshold downstream message processing capability of Openflow Java. Downsizing/upsizing resources surrounding this counter will decrease/increase downstream message processing capability.
Implementation implications:
This new count will be implemented using existing counter module StatisticsCounters.
Please share your comments (if any ).
Thanks and Regards, Chinmay @ ericsson MichalPolkoráb Software Engineer
MichalPolkoráb Software Engineer
MichalPolkoráb Software Engineer
|
|
Re: New message count for Openflow Java in ODL master (Boron)
Chinmay Kumar Mohanta <chinmay.kumar.mohanta@...>
Hi Michal,
From our ongoing effort to debug some of the performance related issues we find that it greatly simplifies the debugging process if error cases (for example like message drop) are reported explicitly rather than implicitly (as a derivation). Explicit error reporting provides *** sensible & precise contextual*** information which will expedite the process of finding the root cause of the issue.
In line with this principle we are still strongly considering to implement this new counter.
Pleases comment?
Thanks and Regards, Chinmay Mohanta
From: Michal Polkoráb [mailto:michal.polkorab@...]
Hi Chinmay,
your point makes sense. However, we should thoroughly consider removing these counters. These were introduced in the past to check if / where there are messages being lost between openflowplugin & openflowjava + to verify the backpressure mechanism. If we similar issue appears again, these counters might be handy.
Michal From: Chinmay.Mohanta <mohanta.chinmay@...>
Hi Michal,
Thanks for your reply.
Indeed the formula indicated by you gives the expected value !! So it obviates the implementation of a new counter.
However following new thoughts emerge.
Do we really need the counts DS_ENTERED_OFJAVA and DS_ENCODE_SUCCESS ?
Maintaining these two counts (for every message) is costly in terms of performance (due to need for synchronization).
I think we should have counts only for the error scenarios resulting least influence on the performance.
BTW, my goal is to simplify the debugging process to find out whenever/wherever there is a message drop due to resource contention. Message drop count will provide an essential input to rationalize the provisioning of resources.
Thanks and Regards, Chinmay @ ericsson
On Wed, May 11, 2016 at 5:22 PM, Michal Polkoráb <michal.polkorab@...> wrote:
Hello Chinmay,
is it really needed to add such counter ? One can get this number by DS_ENTERED_OFJAVA - (DS_ENCODE_SUCCESS + DS_ENCODE_FAIL) if only OutboundQueue is used. This counter looses its meaning when OutboundQueueManager is used as the manager enqueues all messages and the message count is controlled byt rpcQuota in openflowplugin.
Regards, Michal
From: Chinmay.Mohanta <mohanta.chinmay@...>
Hi all,
To supplement present testing and diagnosing scenarios using message counts, I am proposing following new count.
Count name : DS_DROPPED_PACKET_QFULL Purpose : To count messages dropped because the queue present in ChannelOutboundQueue is full. This count will be helpful to mark the threshold downstream message processing capability of Openflow Java. Downsizing/upsizing resources surrounding this counter will decrease/increase downstream message processing capability.
Implementation implications:
This new count will be implemented using existing counter module StatisticsCounters.
Please share your comments (if any ).
Thanks and Regards, Chinmay @ ericsson MichalPolkoráb Software Engineer
MichalPolkoráb Software Engineer
|
|
Re: New message count for Openflow Java in ODL master (Boron)
Michal Polkoráb <michal.polkorab@...>
Hi Chinmay,
you can turn the counters on / off by providing the <statistics> element in 45-openflowjava-stats.xml file (inside openflowjava-config module). If this element is present, counters will be enabled. If the tag is missing, counters won't be enabled.
Regards, Michal From: Chinmay.Mohanta <mohanta.chinmay@...>
Sent: 16 May 2016 08:11 To: Michal Polkoráb Cc: openflowjava-dev@...; shuva.jyoti.kar@...; chinmay.kumar.mohanta@...; ashutosh.bisht@...; muthukumaran.k@... Subject: Re: [openflowjava-dev] New message count for Openflow Java in ODL master (Boron) Hi Michal and all,
In that case I think best would be to introduce the functionality that would allow to enable/disable (via JMX or Config subsys) the counter module on demand.
In this way we can have the message counts when we need it otherwise count collection can be turned off.
comment ?
Thanks and Regards,
Chinmay @ Ericsson
On Fri, May 13, 2016 at 3:37 PM, Michal Polkoráb
<michal.polkorab@...> wrote:
MichalPolkoráb Software Engineer
|
|
Re: New message count for Openflow Java in ODL master (Boron)
Chinmay.Mohanta <mohanta.chinmay@...>
Hi Michal and all, In that case I think best would be to introduce the functionality that would allow to enable/disable (via JMX or Config subsys) the counter module on demand. In this way we can have the message counts when we need it otherwise count collection can be turned off. comment ? Thanks and Regards, Chinmay @ Ericsson On Fri, May 13, 2016 at 3:37 PM, Michal Polkoráb <michal.polkorab@...> wrote:
|
|
Re: New message count for Openflow Java in ODL master (Boron)
Michal Polkoráb <michal.polkorab@...>
Hi Chinmay,
your point makes sense. However, we should thoroughly consider removing these counters. These were introduced in the past to check if / where there are messages being lost between openflowplugin & openflowjava + to verify the backpressure mechanism. If
we similar issue appears again, these counters might be handy.
Michal From: Chinmay.Mohanta <mohanta.chinmay@...>
Sent: 12 May 2016 13:27 To: Michal Polkoráb Cc: openflowjava-dev@...; shuva.jyoti.kar@...; chinmay.kumar.mohanta@...; ashutosh.bisht@...; muthukumaran.k@... Subject: Re: [openflowjava-dev] New message count for Openflow Java in ODL master (Boron) Hi Michal,
Thanks for your reply.
Indeed the formula indicated by you gives the expected value !!
So it obviates the implementation of a new counter.
However following new thoughts emerge.
Do we really need the counts DS_ENTERED_OFJAVA and DS_ENCODE_SUCCESS ?
Maintaining these two counts (for every message) is costly in terms of performance (due to need for synchronization).
I think we should have counts only for the error scenarios resulting least influence on the performance.
BTW, my goal is to simplify the debugging process to find out whenever/wherever there is a message drop due to resource contention. Message drop count will provide an essential input to rationalize the provisioning of resources.
Thanks and Regards,
Chinmay @ ericsson
On Wed, May 11, 2016 at 5:22 PM, Michal Polkoráb
<michal.polkorab@...> wrote:
MichalPolkoráb Software Engineer
|
|
Re: [opendaylight-dev] [OpenDaylight Discuss] TWS on reorganizing the OpenFlow projects
Anil Vishnoi
On Thu, May 12, 2016 at 6:53 AM, Alexis de Talhouët <adetalhouet@...> wrote:
+1
+1
Thanks Anil |
|
Re: [opendaylight-dev] [OpenDaylight Discuss] TWS on reorganizing the OpenFlow projects
Abhijit Kumbhare
Yes - as Alex mentioned during the last meeting the topic was phased release & semantic versioning - but we could do it on the coming Monday. About the last point - I think we should: 1. Not get rid of the existing abstracted OpenDaylight forwarding model in the near future - may be for a long time - as the existing OpenDaylight applications are written toward this forwarding model. 2. Over period add an additional optimized non-abstract OpenFlow forwarding model for select cases where the performance would be critical (example flow programming). Over time applications could migrate toward this forwarding model & then could be deprecated. On Thu, May 12, 2016 at 6:54 AM, Colin Dixon <colin@...> wrote:
|
|
Re: [OpenDaylight Discuss] [opendaylight-dev] TWS on reorganizing the OpenFlow projects
Colin Dixon
And the need to have a new groupID means that downstream projects would be affected and have to change their dependency information in their pom files. --ColinOn Thu, May 12, 2016 at 9:59 AM, Andrew Grimberg <agrimberg@...> wrote: On 05/12/2016 06:43 AM, Colin Dixon wrote: |
|
Re: [opendaylight-dev] [OpenDaylight Discuss] TWS on reorganizing the OpenFlow projects
Andrew Grimberg <agrimberg@...>
On 05/12/2016 06:43 AM, Colin Dixon wrote:
So, did the TWS on this topic actually happen? I think it didn't, but itI'll point out a couple of things with a restructure: 1) Build history might potentially be lost if the projects change their final component name (which is unlikely). Just changing the logical parentage shouldn't be an issue though we would have to lock access to the repos during the transition. 2) We base the artifact namespacing upon the repo name. So, an example of foo/bar/baz.git in Gerrit would have a namespace of org.opendaylight.foo.bar.baz.* in Nexus. 3) With #2 in mind a reorg does mean that their merge jobs will fail until they update their groupID to the newly designated groupID. -Andy- -- Andrew J Grimberg Systems Administrator Release Engineering Team Lead The Linux Foundation |
|
Re: [opendaylight-dev] [OpenDaylight Discuss] TWS on reorganizing the OpenFlow projects
Colin Dixon
The last question is basically, should we merge openflowjava and openflowplugin into one project? leave them apart? or pick a different splitting point? That was basically the focus of the 3-5 mails before mine above.On Thu, May 12, 2016 at 9:53 AM, Alexis de Talhouët <adetalhouet@...> wrote:
|
|
Re: [opendaylight-dev] [OpenDaylight Discuss] TWS on reorganizing the OpenFlow projects
Alexis de Talhouët <adetalhouet@...>
Correct it didn’t, last TWS focused on phased release and semantic versioning. +1, absolutely. +1, those are openflow-related projects, and are tied to openflow. I believe it makes sense having them under the same umbrella. I’m not sure to get the question here so I will at careful at further discussion. Thanks, Alexis
|
|
Re: [opendaylight-dev] [OpenDaylight Discuss] TWS on reorganizing the OpenFlow projects
Colin Dixon
So, did the TWS on this topic actually happen? I think it didn't, but it probably should in the near future. It sounds like there's two (or maybe three) orthogonal questions here: 1.) Should the tightly entwined OpenFlow projects be collocated under a single openflow project like we did with integration and releng? I'd say that's pretty clearly a yes, but we need to decide when and what impact it will have. Thanh and Andy, do you have any comments there? Also, I don't remember if we got to keep group-level isolation of artifact names after we do that, which would be important to understand. 2.) Do we want to move related, but not-so-entwined projects into that too? These would be OF config and TTP for now. I'd argue yes, but I'm not going to fight a war for it. 3.) Do we want to keep the separation between the OpenDaylight forwarding model and OpenFlow plugin? This is more involved and I think could be punted forward, but my general take is that the division we actually wound up with was different than that and maybe less useful. On Mon, May 9, 2016 at 11:42 PM, Anil Vishnoi <vishnoianil@...> wrote:
|
|
Re: New message count for Openflow Java in ODL master (Boron)
Chinmay.Mohanta <mohanta.chinmay@...>
Hi Michal, Thanks for your reply. Indeed the formula indicated by you gives the expected value !! So it obviates the implementation of a new counter. However following new thoughts emerge. Do we really need the counts DS_ENTERED_OFJAVA and DS_ENCODE_SUCCESS ? Maintaining these two counts (for every message) is costly in terms of performance (due to need for synchronization). I think we should have counts only for the error scenarios resulting least influence on the performance. BTW, my goal is to simplify the debugging process to find out whenever/wherever there is a message drop due to resource contention. Message drop count will provide an essential input to rationalize the provisioning of resources. Thanks and Regards, Chinmay @ ericsson On Wed, May 11, 2016 at 5:22 PM, Michal Polkoráb <michal.polkorab@...> wrote:
|
|
Re: New message count for Openflow Java in ODL master (Boron)
Michal Polkoráb <michal.polkorab@...>
Hello Chinmay,
is it really needed to add such counter ? One can get this number by DS_ENTERED_OFJAVA - (DS_ENCODE_SUCCESS + DS_ENCODE_FAIL) if only OutboundQueue is used. This counter looses its meaning when OutboundQueueManager is used as the manager enqueues all messages and the message count is controlled byt rpcQuota in openflowplugin.
Regards, Michal From: Chinmay.Mohanta <mohanta.chinmay@...>
Sent: 10 May 2016 15:22 To: openflowjava-dev@... Cc: shuva.jyoti.kar@...; chinmay.kumar.mohanta@...; ashutosh.bisht@...; muthukumaran.k@... Subject: [openflowjava-dev] New message count for Openflow Java in ODL master (Boron) Hi all,
To supplement present testing and diagnosing scenarios using message counts, I am proposing following new count.
Count name : DS_DROPPED_PACKET_QFULL
Purpose : To count messages dropped because the queue present in ChannelOutboundQueue is full. This count will be helpful to mark the threshold downstream message processing capability of Openflow Java. Downsizing/upsizing resources surrounding this counter
will decrease/increase downstream message processing capability.
Implementation implications:
This new count will be implemented using existing counter module StatisticsCounters.
Please share your comments (if any ).
Thanks and Regards,
Chinmay @ ericsson
MichalPolkoráb Software Engineer
|
|
Re: Hi, question about "flow"
Ryan Dietrich
So, to answer my own question, the message ends up going to Akka, hence the lack of seeing it in the traceback. I knew Akka was in the mix after looking at various diagrams and seeing it all over the logs, but I wasn’t totally sure where the handoff was. On the flip side, when it comes to tracing out calls to the southbound plugin I’m working on, I found that Akka was not in the mix, and instead it went to an LMAX Disruptor against a thread pool (dispatched by the NotificationPopListener/DOMNotificationRouter until eventually the magical BindingDOMNotiificationListenerAdapter does some weird looking reflection call to call to call whatever your plugin is doing). Oh, and I was surprised to see disruptor in the mix, I haven’t seen anyone use that approach since my time in HFT. You guys are serious about performance. So, in case anyone gets this message in a google search, read this document: https://github.com/opendaylight/docs/blob/master/manuals/developer-guide/src/main/asciidoc/openflowjava/odl-openflowjava-protocol-dev.adoc (it is really good, and explains the life cycle and flow of a southbound plugin, specifically the section called: TCP Channel Pipeline). I’m still looking for a hook in my southbound plugin to receive “PacketInMessage” callbacks when they are received by the TCPHandler on 6653. Today I’m going to randomly try assigning various elements to my yang configuration to see if the appropriate structure shows up in my auto-generated “Abstract${PROJECT}ImplModule.java” file. Again, if anyone can give me a hint or a clue I’d appreciate it. -Ryan Dietrich
|
|
New message count for Openflow Java in ODL master (Boron)
Chinmay.Mohanta <mohanta.chinmay@...>
Hi all, To supplement present testing and diagnosing scenarios using message counts, I am proposing following new count. Count name : DS_DROPPED_PACKET_QFULL Purpose : To count messages dropped because the queue present in ChannelOutboundQueue is full. This count will be helpful to mark the threshold downstream message processing capability of Openflow Java. Downsizing/upsizing resources surrounding this counter will decrease/increase downstream message processing capability. Implementation implications: This new count will be implemented using existing counter module StatisticsCounters. Please share your comments (if any ). Thanks and Regards, Chinmay @ ericsson |
|