Faseela K <faseela.k@...>
Michael,
The shard list is being dumped to make sure our single shard configuration is proper at different places of the CSIT. Not to check if cluster is ready.
Thanks,
Faseela
toggle quoted messageShow quoted text
-----Original Message----- From: Jamo Luhrsen [mailto:jluhrsen@...] Sent: Wednesday, July 04, 2018 3:31 AM To: Michael Vorburger <vorburger@...>; Faseela K <faseela.k@...> Cc: Sam Hague <shague@...>; aaa-dev@...; odl netvirt dev <netvirt-dev@...> Subject: Re: [Aaa-dev] [netvirt-dev] Random errors while fetching JOLOKIA URLs in 3 node CSIT On 07/03/2018 12:31 PM, Michael Vorburger wrote: On Tue, Jul 3, 2018 at 7:21 AM, Faseela K <faseela.k@... <mailto:faseela.k@...>>wrote:
Tom,____
I couldn’t follow your response below completely.____
But, the URL whatever I am using in CSIT works most of the times, and only in some random cases it throws the
shiro error.____
When I updated the CSIT to retry on failure, it works on
subsequent retries.____
Below is the exception stack trace :
I don't know the details of what this CSIT wants from
/jolokia/read/org.opendaylight.controller:type=DistributedOperationalD
atastore,Category=ShardManager,name=shard-manager-operational/LocalSha
rds, but I don't suppose it could (now) be replaced with GET
/diagstatus from https://jira.opendaylight.org/browse/GENIUS-138
? That requires no authentication, so it would never suffer from this particular problem. Just a thought. even if that is a better thing for CSIT to do, it would just be a workaround to whatever problem is really there. JamO ____
2018-07-02T02:47:54,463 | INFO |
org.ops4j.pax.web.service.jetty.internal.LateInvalidatingHashSessionManager@38d77047Timer |
LateInvalidatingHashSessionManager | 438 - org.ops4j.pax.web.pax-web-jetty - 6.0.9 | Timing out for 1 session(s)
with id 3u5gk3qqbpby1c13f3hpivnge____
2018-07-02T02:47:54,470 | INFO |
org.ops4j.pax.web.service.jetty.internal.LateInvalidatingHashSessionManager@19c0aeb1Timer |
LateInvalidatingHashSessionManager | 438 - org.ops4j.pax.web.pax-web-jetty - 6.0.9 | Timing out for 1 session(s)
with id yev7cqw2kyjsy6978sf1nmrl____
2018-07-02T02:48:05,934 | WARN | qtp1371496041-116 | HttpChannel | 156 - org.eclipse.jetty.util -
9.3.21.v20170918 |
//10.30.170.46:8181/jolokia/read/org.opendaylight.controller:type=DistributedOperationalDatastore,Category=ShardManager,name=shard-manager-operational/LocalShards
<http://10.30.170.46:8181/jolokia/read/org.opendaylight.controller:typ
e=DistributedOperationalDatastore,Category=ShardManager,name=shard-man
ager-operational/LocalShards>____
org.apache.shiro.session.UnknownSessionException: There is no
session with id [bf682403-082b-4520-993f-987a8804bd31]____
at org.apache.shiro.session.mgt.eis.AbstractSessionDAO.readSession(AbstractSessionDAO.java:170)
[134:org.apache.shiro.core:1.3.2]____
at
org.apache.shiro.session.mgt.DefaultSessionManager.retrieveSessionFromDataSource(DefaultSessionManager.java:236)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.session.mgt.DefaultSessionManager.retrieveSession(DefaultSessionManager.java:222)
[134:org.apache.shiro.core:1.3.2]____
at
org.apache.shiro.session.mgt.AbstractValidatingSessionManager.doGetSes
sion(AbstractValidatingSessionManager.java:118)
[134:org.apache.shiro.core:1.3.2]____
at
org.apache.shiro.session.mgt.AbstractNativeSessionManager.lookupSession(AbstractNativeSessionManager.java:148)
[134:org.apache.shiro.core:1.3.2]____
at
org.apache.shiro.session.mgt.AbstractNativeSessionManager.lookupRequiredSession(AbstractNativeSessionManager.java:152)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.session.mgt.AbstractNativeSessionManager.getHost(AbstractNativeSessionManager.java:237)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.session.mgt.DelegatingSession.getHost(DelegatingSession.java:111)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.session.ProxiedSession.getHost(ProxiedSession.java:93)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.session.ProxiedSession.getHost(ProxiedSession.java:93)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.subject.support.DefaultSubjectContext.resolveHost(DefaultSubjectContext.java:270)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.mgt.DefaultSubjectFactory.createSubject(DefaultSubjectFactory.java:45)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.mgt.DefaultSecurityManager.doCreateSubject(DefaultSecurityManager.java:369)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.mgt.DefaultSecurityManager.createSubject(DefaultSecurityManager.java:344)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.mgt.DefaultSecurityManager.createSubject(DefaultSecurityManager.java:183)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:283)
[134:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:256)
[134:org.apache.shiro.core:1.3.2]____
at org.opendaylight.aaa.authenticator.ODLAuthenticator.authenticate(ODLAuthenticator.java:42)
[198:org.opendaylight.aaa.shiro:0.8.0.SNAPSHOT]____
at
Proxyb44c2d91_1f07_4b19_be55_2f3cb0209c21.authenticate(Unknown Source)
[?:?]____
at
org.jolokia.osgi.security.ServiceAuthenticationHttpContext.handleSecurity(ServiceAuthenticationHttpContext.java:72)
[189:org.jolokia.osgi:1.5.0]____
at
org.ops4j.pax.web.service.internal.WebContainerContextWrapper.handleSecurity(WebContainerContextWrapper.java:46)
[440:org.ops4j.pax.web.pax-web-runtime:6.0.9]____
at
org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:69)
[438:org.ops4j.pax.web.pax-web-jetty:6.0.9]____
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
[153:org.eclipse.jetty.server:9.3.21.v20170918]____
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
[151:org.eclipse.jetty.security:9.3.21.v20170918]____
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
[153:org.eclipse.jetty.server:9.3.21.v20170918]____
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[153:org.eclipse.jetty.server:9.3.21.v20170918]____
at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:284)
[438:org.ops4j.pax.web.pax-web-jetty:6.0.9]____
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[154:org.eclipse.jetty.servlet:9.3.21.v20170918]____
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[153:org.eclipse.jetty.server:9.3.21.v20170918]____
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[153:org.eclipse.jetty.server:9.3.21.v20170918]____
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[153:org.eclipse.jetty.server:9.3.21.v20170918]____
at
org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80)
[438:org.ops4j.pax.web.pax-web-jetty:6.0.9]____
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[153:org.eclipse.jetty.server:9.3.21.v20170918]____
at
org.eclipse.jetty.server.Server.handle(Server.java:534)
[153:org.eclipse.jetty.server:9.3.21.v20170918]____
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
[153:org.eclipse.jetty.server:9.3.21.v20170918]____
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[153:org.eclipse.jetty.server:9.3.21.v20170918]____
at org.eclipse.jetty.io
<http://org.eclipse.jetty.io>.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[145:org.eclipse.jetty.io:9.3.21.v20170918]____
at org.eclipse.jetty.io <http://org.eclipse.jetty.io>.FillInterest.fillable(FillInterest.java:108)
[145:org.eclipse.jetty.io:9.3.21.v20170918]____
at org.eclipse.jetty.io
<http://org.eclipse.jetty.io>.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[145:org.eclipse.jetty.io:9.3.21.v20170918]____
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[156:org.eclipse.jetty.util:9.3.21.v20170918]____
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[156:org.eclipse.jetty.util:9.3.21.v20170918]____
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[156:org.eclipse.jetty.util:9.3.21.v20170918]____
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[156:org.eclipse.jetty.util:9.3.21.v20170918]____
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[156:org.eclipse.jetty.util:9.3.21.v20170918]____
at java.lang.Thread.run(Thread.java:748) [?:?]____
__ __
____
Thanks,____
Faseela____
__ __
*From:*Tom Pantelis [mailto:tompantelis@... <mailto:tompantelis@...>]
*Sent:* Monday, July 02, 2018 10:29 PM
*To:* Sam Hague <shague@... <mailto:shague@...>>
*Cc:* Vishal Thapar <vthapar@... <mailto:vthapar@...>>; odl netvirt dev
<netvirt-dev@... <mailto:netvirt-dev@...>>; aaa-dev@...
<mailto:aaa-dev@...>; Faseela K <faseela.k@... <mailto:faseela.k@...>>
*Subject:* Re: [Aaa-dev] [netvirt-dev] Random errors while
fetching JOLOKIA URLs in 3 node CSIT____
__ __
jolokia has a different security code path than restconf that
interfaces with shiro. It starts with:____
__ __
....____
at org.apache.shiro.subject.support.DefaultSubjectContext.resolveHost(DefaultSubjectContext.java:270)
[130:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.mgt.DefaultSubjectFactory.createSubject(DefaultSubjectFactory.java:45)
[130:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.mgt.DefaultSecurityManager.doCreateSubject(DefaultSecurityManager.java:369)
[130:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.mgt.DefaultSecurityManager.createSubject(DefaultSecurityManager.java:344)
[130:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.mgt.DefaultSecurityManager.createSubject(DefaultSecurityManager.java:183)
[130:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:283)
[130:org.apache.shiro.core:1.3.2]____
at org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:256)
[130:org.apache.shiro.core:1.3.2]____
at org.opendaylight.aaa.authenticator.ODLAuthenticator.authenticate(ODLAuthenticator.java:42)
[188:org.opendaylight.aaa.shiro:0.8.0.SNAPSHOT]____
at
Proxy02007817_e19a_4a1a_8178_e2f238fd17a0.authenticate(Unknown Source)
[?:?]____
at
org.jolokia.osgi.security.ServiceAuthenticationHttpContext.handleSecurity(ServiceAuthenticationHttpContext.java:72)
[179:org.jolokia.osgi:1.5.0]____
at
org.ops4j.pax.web.service.internal.WebContainerContextWrapper.handleSecurity(WebContainerContextWrapper.java:46)
[312:org.ops4j.pax.web.pax-web-runtime:6.0.9]____
at
org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:69)
[310:org.ops4j.pax.web.pax-web-jetty:6.0.9]____
__ __
__ __
On Mon, Jul 2, 2018 at 12:44 PM, Sam Hague <shague@...
<mailto:shague@...>> wrote:____
Could this be related to the clustering bugs Vic and Jamo are
following like [1] that deal with 500 responses?____
__ __
[1] https://jira.opendaylight.org/browse/CONTROLLER-1838
<https://jira.opendaylight.org/browse/CONTROLLER-1838>____
__ __
On Mon, Jul 2, 2018 at 2:27 AM Vishal Thapar
<vthapar@... <mailto:vthapar@...>> wrote:____
I've seen this in my local testing during bringup or during ownership change. Sometimes it persists even
post ownership change.____
__ __
On Mon, Jul 2, 2018 at 11:24 AM, Faseela K
<faseela.k@... <mailto:faseela.k@...>> wrote:____
Hello aaa-dev,____
I am trying to dump some of the MDSAL exposed MBeans with jolokia over HTTP , while am experimenting
with some of the 3 node netvirt CSIT activities
[0].____
The HTTP requests work most of the cases, but
sometimes hit the below error randomly?____
Any pointers?____
____
<html> <head> <meta http-equiv="Content-Type" content="text/html; <title>Error 500 </title> </head>
<body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing
/jolokia/read/org.opendaylight.controller:type=DistributedOperationalDatastore,Category=ShardManager,name=shard-manager-operational/LocalShards.
Reason: <pre> org.apache.shiro.session.UnknownSessionException: There is no session with id
[4050ca12-d12d-4965-8204-d3c11cb16e1b]</pre></p> <hr
/> </body> </html>____
____
____
Thanks,____
Faseela____
____
[0]
https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/faseelanetvirt-csit-3node-openstack-queens-upstream-stateful-fluorine/1/robot-plugin/log_full.html.gz
<https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/faseelane
tvirt-csit-3node-openstack-queens-upstream-stateful-fluorine/1/robot-p
lugin/log_full.html.gz>____
____
_______________________________________________
netvirt-dev mailing list
netvirt-dev@... <mailto:netvirt-dev@...>
https://lists.opendaylight.org/mailman/listinfo/netvirt-dev
<https://lists.opendaylight.org/mailman/listinfo/netvirt-dev>____
__ __
_______________________________________________
netvirt-dev mailing list
netvirt-dev@... <mailto:netvirt-dev@...>
https://lists.opendaylight.org/mailman/listinfo/netvirt-dev
<https://lists.opendaylight.org/mailman/listinfo/netvirt-dev>____
_______________________________________________
aaa-dev mailing list
aaa-dev@... <mailto:aaa-dev@...>
https://lists.opendaylight.org/mailman/listinfo/aaa-dev
<https://lists.opendaylight.org/mailman/listinfo/aaa-dev>____
__ __
_______________________________________________
netvirt-dev mailing list
netvirt-dev@... <mailto:netvirt-dev@...>
https://lists.opendaylight.org/mailman/listinfo/netvirt-dev
<https://lists.opendaylight.org/mailman/listinfo/netvirt-dev>
_______________________________________________
aaa-dev mailing list
aaa-dev@...
https://lists.opendaylight.org/mailman/listinfo/aaa-dev
|
