Vishal Thapar <vishal.thapar@...>
Aha! I meant for Manager. This field is present in Manager and Controller both and I was specifically talking about Manager. For controller,
we were also creating controller manually, not using the autobridge code. I’d recommend creating an enhancement bug for this. We should add this knob and code to autobridge.
Regards,
Vishal.
toggle quoted message
Show quoted text
From: Sela, Guy [mailto:guy.sela@...]
Sent: 17 January 2017 16:23
To: Vishal Thapar <vishal.thapar@...>; Pearl, Tomer <tomer.pearl@...>; Anil Vishnoi <vishnoianil@...>
Cc: Muthukumaran K <muthukumaran.k@...>; Jamo Luhrsen <jluhrsen@...>; marcus.g.williams@...; openflowjava-dev@...; ovsdb-dev@...
Subject: RE: [openflowjava-dev] [ovsdb-dev] OVSDB scale
When you’re saying initial configuration do you mean before the OVS established an openflow connection?
Configuration for Controller Table in OVSDB is being set by ODL.
The CLI configuration for the inactivity probe looks like this for example:
sudo ovs-vsctl add Controller 8383a19f-4899-4808-ba0b-c970af081c3e inactivity_probe 10000
So it looks like this can only be set after the connection
We don’t have configuration for this today. We can add it or change netvirt code to add API. We changed it directly on OVS as part of initial configuration. We
were using scripts to configure OVSes with manager, so just added one more command. Another parameter you may want to look at is stats_interval which governs how frequently stat updates come, though we later on disabled stats by default.
Finally, if you’re using HA Proxy between OVS and ODL for manager connection or using single node i.e. OVS connects to only one manager at a time, you can tweak
a flag captured in this:
https://git.opendaylight.org/gerrit/#/c/49068/
If you change this for a deployment where each OVS connects to each ODL node in cluster [multiple manager connections to cluster] changing this flag can have
functional impact, so be careful.
Regards,
Vishal.
Can we set this value via OVSDB configuration file? Or do we need to change code to use some api for this?
Thanks
Close, inactivity_probe. Sorry for delay, had to fish in old logs.
max_backoff ?
Great thanks.
Do you recall where you tweak this configuration ? A quick google search didn’t help me.
Good catch Anil. I forgot that we *did* increased timeout to 30-60 seconds from default of 5. I say 30-60 because we did different testing to reduce no.
of echo messages going back and forth. But with 5 we used to see frequent disconnects, so yes, I’d agree with Anil that increasing timeouts should be a better solution. In fact default of 5 is terrible as you start scaling up, you’ll be processing too many
echo messages.
Regards,
Vishal.
Full GC in a 8G-16G Heap takes about 10+ seconds
From: Anil Vishnoi [mailto:vishnoianil@...]
Sent: Tuesday, January 17, 2017 11:25 AM
To: Sela, Guy <guy.sela@...>
Cc: Vishal Thapar <vishal.thapar@...>; Muthukumaran K <muthukumaran.k@...>; Jamo Luhrsen <jluhrsen@...>;
Pearl, Tomer <tomer.pearl@...>;
marcus.g.williams@...;
openflowjava-dev@...;
ovsdb-dev@...
Subject: Re: [openflowjava-dev] [ovsdb-dev] OVSDB scale
I think we should look at why OVS is getting disconnected during the GC? Is it because of the Echo timeout? Tuning GC will help, but i don't think so it will fix the root cause. I think if we can increase the echo timeouts,
probably disconnection won't happen atleast because of GC.
On Tue, Jan 17, 2017 at 1:15 AM, Sela, Guy <guy.sela@...> wrote:
Hi guys,
So a couple of questions:
1)
Did you reached Full GC? And if so, did the OVSs disconnected? And did everything continued working smoothly afterwards?
2)
Do you have some script or mechanism you can share that will allow to quickly count number of flows in the data store?
Hi Muthu,
Yeah, we had used similar heap and GC settings when testing ITM which added OVSDB and all the netvirt
models to the mix, I couldn’t recall what exactly were those.
Regards,
Vishal.
Hi Vishal,
We had focused mainly on baseline feature of OFPlugin so some of our changes specific to that drive-test
may not be applicable for Guy’s case. However, increasing the Heap and using G1GC is something he had already accounted.
For the scenario we were chasing (only openflowplugin + a load-driver app – bulk-o-matic), we had used
the settings mentioned in the last reply of this bug
https://bugs.opendaylight.org/show_bug.cgi?id=4823
There are few more tweaks in Openflowplugin – but they are all related to specifics of OFPlugin (Helium)
Regards
Muthu
Adding Muthu.
Muthu,
I believe we did have to do some tweaks with heapsize, GC settings etc.right? Do you recall?
Regards,
Vishal.
Thanks,
Did you manage to survive Full GCs at all ?
If I don’t avoid it, a Full GC causes all OVSs to disconnect from the ODL, and it results
in a bit of chaos. Is there any way around this other than avoiding Full GC? I managed to avoid it in my testing using 16G heap size and G1 collector.
We tested not just OVSDB but OVSDB+Netvirt/VPNService at scale of about 80 OVS at the time with a full
mesh. Scale limits come more from size of datastore than anything else. So how many devices you can scale depends on extent of features you’re testing. Is it just OVSDB, or Netvit with multiple VMs per compute across multiple networks?
If you’re running into memory issues would be good to increase memory and capture memory usage. While
provisioning you may hit a high peak but will come down once it is done. I’ll check if I can get details of numbers we tested, should be lying somewhere in archived mails.
Regards,
Vishal.
From: Anil
Vishnoi [mailto:vishnoianil@...]
Sent: 17 January 2017 12:09
To: Jamo Luhrsen <jluhrsen@...>
Cc: Pearl, Tomer <tomer.pearl@...>;
marcus.g.williams@...;
openflowjava-dev@...;
ovsdb-dev@...; Sela, Guy <guy.sela@...>; Vishal Thapar <vishal.thapar@...>
Subject: Re: [openflowjava-dev] [ovsdb-dev] OVSDB scale
I believe team from Ericsson also did some testing with it and we made some more performance improvement on boron.
@vishal : do you have any number from your ovsdb testing ?
On Tue, Jan 3, 2017 at 10:05 PM, Jamo Luhrsen <jluhrsen@...> wrote:
Hi Tomer,
back in Beryllium there was a performance report released [0]. You can see on page 31 that we
saw OVSDB scale up to 1800 nodes. There may be more recent tests done, and I think Marcus
may have some idea. But, I think your 200 number should be achievable.
Thanks,
JamO
[0]
https://www.opendaylight.org/sites/www.opendaylight.org/files/odl_performancetechnicalreport_1-1_052716.pdf
On 01/02/2017 06:02 AM, Pearl, Tomer wrote:
> Hi,
>
>
>
> I’m trying to bring up a setup with one ODL controller and 200+ OVSs.
>
> I’m testing with Boron SR1 code
>
>
>
> Are there any reports about ODL scale tests that I can look at ?
>
> Is 200 OVSs an amount that supposed to work?
>
>
>
> Thanks,
>
>
>
> Tomer P.
>
>
>
>
>
>
>
>
>
> _______________________________________________
> ovsdb-dev mailing list
> ovsdb-dev@...
>
https://lists.opendaylight.org/mailman/listinfo/ovsdb-dev
>
_______________________________________________
openflowjava-dev mailing list
openflowjava-dev@...
https://lists.opendaylight.org/mailman/listinfo/openflowjava-dev
--
--
|
