[netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".


Faseela K <k.faseela@...>
 

Srinivas,
   As you have raised NETVIRT-1640 to track the issue, you can add the exception to the CSIT whitelist, so that CSIT is unblocked.
   We can remove it from the list whenever issue is fixed.
Thanks,
Faseela

On Thu, 14 Nov 2019 at 23:39, Luis Gomez <ecelgp@...> wrote:
Also looping netvirt list, to see how they want to proceed here.

On Nov 14, 2019, at 10:06 AM, Luis Gomez via Lists.Opendaylight.Org <ecelgp=gmail.com@...> wrote:

Thanks Jamo, I guess if netvirt project is interested in the JVM framework they can patch as you suggest, otherwise we can just disable it for netvirt so we have less things to maintain in integration.

BR/Luis


On Nov 14, 2019, at 9:53 AM, Jamo Luhrsen <jluhrsen@...> wrote:

or you can add the exception to this list:


https://github.com/opendaylight/integration-test/blob/master/csit/libraries/netvirt/excepts.py


eventually, assuming magnesium starts to work with decanter, you will hopefully
get the jvm monitoring.

JamO


On 11/14/19 9:44 AM, Luis Gomez wrote:
The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring.

[1] https://git.opendaylight.org/gerrit/#/c/releng/builder/+/85756/

BR/Luis

On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

This is one possibility.
 
Can someone more experienced than me to see this recomendation about elasticsearch:
Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:
Client client = new TransportClient()
                .addTransportAddress(new InetSocketTransportAddress(
                        "143.79.236.xxx",
                        9300));
 
Maybe it is also our problem....
Stan
 
From: Srinivas Rachakonda <srinivas.rachakonda@...> 
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi, 
 
Can the changes be reverted back as many CSIT jobs are failing. 
 
 

From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support 

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here 

https://jenkins.opendaylight.org/sandbox/job/stan-JVMdisable-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/1/

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

https://jenkins.opendaylight.org/sandbox/job/stan-fixed-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/ - with JVM support

https://jenkins.opendaylight.org/sandbox/job/stanislav-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/2/ - without JVM support

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...> 
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...; integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.

 

On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13813): https://lists.opendaylight.org/g/integration-dev/message/13813
Mute This Topic: https://lists.opendaylight.org/mt/55406880/535209
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [jluhrsen@...]
-=-=-=-=-=-=-=-=-=-=-=-


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13815): https://lists.opendaylight.org/g/integration-dev/message/13815
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8403): https://lists.opendaylight.org/g/netvirt-dev/message/8403
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1993029
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [k.faseela@...]
-=-=-=-=-=-=-=-=-=-=-=-


--
Faseela.K


Srinivas <srinivas.rachakonda@...>
 

Hi Faseela,

 

I have raised the below review:

https://git.opendaylight.org/gerrit/c/integration/test/+/85772

 

 

Thanks,

Srinivas

+91-9243478719

 

From: netvirt-dev@... <netvirt-dev@...> On Behalf Of Faseela K
Sent: 15 November 2019 14:03
To: Luis Gomez <ecelgp@...>
Cc: odl netvirt dev <netvirt-dev@...>; Jamo Luhrsen <jluhrsen@...>; integration-dev@...
Subject: Re: [netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Srinivas,

   As you have raised NETVIRT-1640 to track the issue, you can add the exception to the CSIT whitelist, so that CSIT is unblocked.

   We can remove it from the list whenever issue is fixed.

Thanks,

Faseela

 

On Thu, 14 Nov 2019 at 23:39, Luis Gomez <ecelgp@...> wrote:

Also looping netvirt list, to see how they want to proceed here.



On Nov 14, 2019, at 10:06 AM, Luis Gomez via Lists.Opendaylight.Org <ecelgp=gmail.com@...> wrote:

 

Thanks Jamo, I guess if netvirt project is interested in the JVM framework they can patch as you suggest, otherwise we can just disable it for netvirt so we have less things to maintain in integration.

 

BR/Luis

 



On Nov 14, 2019, at 9:53 AM, Jamo Luhrsen <jluhrsen@...> wrote:

 

or you can add the exception to this list:


https://github.com/opendaylight/integration-test/blob/master/csit/libraries/netvirt/excepts.py


eventually, assuming magnesium starts to work with decanter, you will hopefully
get the jvm monitoring.

JamO

On 11/14/19 9:44 AM, Luis Gomez wrote:

The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring.

 

[1] https://git.opendaylight.org/gerrit/#/c/releng/builder/+/85756/

 

BR/Luis

 

On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

This is one possibility.

 

Can someone more experienced than me to see this recomendation about elasticsearch:

Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:

Client client = new TransportClient()

                .addTransportAddress(new InetSocketTransportAddress(

                        "143.79.236.xxx",

                        9300));

 

Maybe it is also our problem....

Stan

 

From: Srinivas Rachakonda <srinivas.rachakonda@...
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi, 

 

Can the changes be reverted back as many CSIT jobs are failing. 

 

 


From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support 

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here 

https://jenkins.opendaylight.org/sandbox/job/stan-JVMdisable-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/1/

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

https://jenkins.opendaylight.org/sandbox/job/stan-fixed-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/ - with JVM support

https://jenkins.opendaylight.org/sandbox/job/stanislav-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/2/ - without JVM support

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.

 

On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
 
View/Reply Online (#13813): https://lists.opendaylight.org/g/integration-dev/message/13813
Mute This Topic: https://lists.opendaylight.org/mt/55406880/535209
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [jluhrsen@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13815): https://lists.opendaylight.org/g/integration-dev/message/13815
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8403): https://lists.opendaylight.org/g/netvirt-dev/message/8403
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1993029
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [k.faseela@...]
-=-=-=-=-=-=-=-=-=-=-=-


 

--

Faseela.K


Luis Gomez
 

FYI I have already merged Stanislav patches that allow projects to opt out from JVM monitoring. This means your CSIT should be working back now.

Stanislav, netvirt project has filed this patch:


Meaning they are still interested in the framework, can you please test it along with the JVM framework? if it works we can re-enable JVM monitoring in the netvirt jobs.

BR/Luis




On Nov 15, 2019, at 1:55 AM, <srinivas.rachakonda@...> <srinivas.rachakonda@...> wrote:

Hi Faseela,
 
I have raised the below review:
 
 
Thanks,
Srinivas
+91-9243478719
 
From: netvirt-dev@... <netvirt-dev@...> On Behalf Of Faseela K
Sent: 15 November 2019 14:03
To: Luis Gomez <ecelgp@...>
Cc: odl netvirt dev <netvirt-dev@...>; Jamo Luhrsen <jluhrsen@...>; integration-dev@...
Subject: Re: [netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Srinivas,
   As you have raised NETVIRT-1640 to track the issue, you can add the exception to the CSIT whitelist, so that CSIT is unblocked.
   We can remove it from the list whenever issue is fixed.
Thanks,
Faseela
 
On Thu, 14 Nov 2019 at 23:39, Luis Gomez <ecelgp@...> wrote:
Also looping netvirt list, to see how they want to proceed here.


On Nov 14, 2019, at 10:06 AM, Luis Gomez via Lists.Opendaylight.Org <ecelgp=gmail.com@...> wrote:
 
Thanks Jamo, I guess if netvirt project is interested in the JVM framework they can patch as you suggest, otherwise we can just disable it for netvirt so we have less things to maintain in integration.
 
BR/Luis
 


On Nov 14, 2019, at 9:53 AM, Jamo Luhrsen <jluhrsen@...> wrote:
 

or you can add the exception to this list:


https://github.com/opendaylight/integration-test/blob/master/csit/libraries/netvirt/excepts.py


eventually, assuming magnesium starts to work with decanter, you will hopefully
get the jvm monitoring.

JamO

On 11/14/19 9:44 AM, Luis Gomez wrote:
The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring. 
 
 
BR/Luis
 
On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:
 
This is one possibility.
 
Can someone more experienced than me to see this recomendation about elasticsearch:
Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:
Client client = new TransportClient()
                .addTransportAddress(new InetSocketTransportAddress(
                        "143.79.236.xxx",
                        9300));
 
Maybe it is also our problem....
Stan
 
From: Srinivas Rachakonda <srinivas.rachakonda@...
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi, 
 
Can the changes be reverted back as many CSIT jobs are failing. 
 
 

From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Well  I was wrong that was the reason.
I tried to run the failing job without JVM support 
(patched with the new patch which I prepared:
Setting the parameter
ELASTICSEARCHATTRIBUTE=disabled
Will disable all JVM related stuff in job)
Here 
the job is without those failures
So I conclude that there is really some connection between JVM support and those failures.
 
I also compared these two outputs for the same job:
 
Currently I do not know what is the reason for faulures or how to fix it...
Stan
 
 
 
 
From: Luis Gomez <ecelgp@...
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.
 
On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:
 
Hi all
I tried to analyze the problem.
I think that it is due to the fact that patch
which removed the explicit mention of stuff related to JVM from scriptplan and testplan
was merged on Nov 12 6:53 PM
Was merged on Nov 11 9:19 PM
 
And this build 
was executed on Nov 12, 2019 11:15:47 AM)
 
We well see if the next build is going to be successfull
 
Stan
 
From: srinivas.rachakonda@... <srinivas.rachakonda@...
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi All,
 
Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.
 
 
 
Yesterday there was a commit in builder file: jjb/integration/common-functions.sh
 
<image001.png>
 
Could this be the cause of failure. 
Please kindly help.
 
Thanks,
Srinivas
+91-9243478719
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-
 
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
 
View/Reply Online (#13813): https://lists.opendaylight.org/g/integration-dev/message/13813
Mute This Topic: https://lists.opendaylight.org/mt/55406880/535209
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [jluhrsen@...]
-=-=-=-=-=-=-=-=-=-=-=-
 
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13815): https://lists.opendaylight.org/g/integration-dev/message/13815
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8403): https://lists.opendaylight.org/g/netvirt-dev/message/8403
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1993029
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [k.faseela@...]
-=-=-=-=-=-=-=-=-=-=-=-

 
-- 
Faseela.K


Srinivas Rachakonda
 

Hi,

I am seeing a new error.

sent 8 bytes  received 87 bytes  190.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1650) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)
mv: cannot stat ‘local.conf_compute_10.30.170.65’: No such file or directory
Warning: Permanently added '10.30.170.79' (ECDSA) to the list of known hosts.
Connection to 10.30.170.79 closed.
tempest results not found in /opt/stack/tempest/.stestr/0
Robot results publisher started...
-Parsing output xml:
Failed!
hudson.AbortException: No files found in path /w/workspace/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-rocky-dcgw-magnesium with configured filemask: output.xml
	at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:78)
	at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:54)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3052)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to prd-centos7-robot-2c-8g-13687
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:957)
		at hudson.FilePath.act(FilePath.java:1072)
		at hudson.FilePath.act(FilePath.java:1061)
		at hudson.plugins.robot.RobotParser.parse(RobotParser.java:49)
		at hudson.plugins.robot.RobotPublisher.parse(RobotPublisher.java:224)
		at hudson.plugins.robot.RobotPublisher.perform(RobotPublisher.java:246)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:79)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1840)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
[PostBuildScript] - Executing post build scripts.
[netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-rocky-dcgw-magnesium] $ /bin/bash /tmp/jenkins198548001739126173.sh
Archiving csit artifacts

R.Srinivas
+91-9243478719



On Fri, 15 Nov 2019 at 22:19, Luis Gomez <ecelgp@...> wrote:
FYI I have already merged Stanislav patches that allow projects to opt out from JVM monitoring. This means your CSIT should be working back now.

Stanislav, netvirt project has filed this patch:


Meaning they are still interested in the framework, can you please test it along with the JVM framework? if it works we can re-enable JVM monitoring in the netvirt jobs.

BR/Luis




On Nov 15, 2019, at 1:55 AM, <srinivas.rachakonda@...> <srinivas.rachakonda@...> wrote:

Hi Faseela,
 
I have raised the below review:
 
 
Thanks,
Srinivas
+91-9243478719
 
From: netvirt-dev@... <netvirt-dev@...> On Behalf Of Faseela K
Sent: 15 November 2019 14:03
To: Luis Gomez <ecelgp@...>
Cc: odl netvirt dev <netvirt-dev@...>; Jamo Luhrsen <jluhrsen@...>; integration-dev@...
Subject: Re: [netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Srinivas,
   As you have raised NETVIRT-1640 to track the issue, you can add the exception to the CSIT whitelist, so that CSIT is unblocked.
   We can remove it from the list whenever issue is fixed.
Thanks,
Faseela
 
On Thu, 14 Nov 2019 at 23:39, Luis Gomez <ecelgp@...> wrote:
Also looping netvirt list, to see how they want to proceed here.


On Nov 14, 2019, at 10:06 AM, Luis Gomez via Lists.Opendaylight.Org <ecelgp=gmail.com@...> wrote:
 
Thanks Jamo, I guess if netvirt project is interested in the JVM framework they can patch as you suggest, otherwise we can just disable it for netvirt so we have less things to maintain in integration.
 
BR/Luis
 


On Nov 14, 2019, at 9:53 AM, Jamo Luhrsen <jluhrsen@...> wrote:
 

or you can add the exception to this list:


https://github.com/opendaylight/integration-test/blob/master/csit/libraries/netvirt/excepts.py


eventually, assuming magnesium starts to work with decanter, you will hopefully
get the jvm monitoring.

JamO

On 11/14/19 9:44 AM, Luis Gomez wrote:
The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring. 
 
 
BR/Luis
 
On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:
 
This is one possibility.
 
Can someone more experienced than me to see this recomendation about elasticsearch:
Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:
Client client = new TransportClient()
                .addTransportAddress(new InetSocketTransportAddress(
                        "143.79.236.xxx",
                        9300));
 
Maybe it is also our problem....
Stan
 
From: Srinivas Rachakonda <srinivas.rachakonda@...
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi, 
 
Can the changes be reverted back as many CSIT jobs are failing. 
 
 

From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Well  I was wrong that was the reason.
I tried to run the failing job without JVM support 
(patched with the new patch which I prepared:
Setting the parameter
ELASTICSEARCHATTRIBUTE=disabled
Will disable all JVM related stuff in job)
Here 
the job is without those failures
So I conclude that there is really some connection between JVM support and those failures.
 
I also compared these two outputs for the same job:
 
Currently I do not know what is the reason for faulures or how to fix it...
Stan
 
 
 
 
From: Luis Gomez <ecelgp@...
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.
 
On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:
 
Hi all
I tried to analyze the problem.
I think that it is due to the fact that patch
which removed the explicit mention of stuff related to JVM from scriptplan and testplan
was merged on Nov 12 6:53 PM
Was merged on Nov 11 9:19 PM
 
And this build 
was executed on Nov 12, 2019 11:15:47 AM)
 
We well see if the next build is going to be successfull
 
Stan
 
From: srinivas.rachakonda@... <srinivas.rachakonda@...
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi All,
 
Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.
 
 
 
Yesterday there was a commit in builder file: jjb/integration/common-functions.sh
 
<image001.png>
 
Could this be the cause of failure. 
Please kindly help.
 
Thanks,
Srinivas
+91-9243478719
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-
 
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
 
View/Reply Online (#13813): https://lists.opendaylight.org/g/integration-dev/message/13813
Mute This Topic: https://lists.opendaylight.org/mt/55406880/535209
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [jluhrsen@...]
-=-=-=-=-=-=-=-=-=-=-=-
 
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13815): https://lists.opendaylight.org/g/integration-dev/message/13815
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8403): https://lists.opendaylight.org/g/netvirt-dev/message/8403
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1993029
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [k.faseela@...]
-=-=-=-=-=-=-=-=-=-=-=-

 
-- 
Faseela.K

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8406): https://lists.opendaylight.org/g/netvirt-dev/message/8406
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1965958
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [srinivas.rachakonda@...]
-=-=-=-=-=-=-=-=-=-=-=-


JamO Luhrsen
 

Looks like stacking failed. Compute node failed because it couldn't
clone the neutron repo:


https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-rocky-dcgw-magnesium/69/control_1/stack.log.gz


Hopefully an intermittent problem in upstream neurton infrastructure.
maybe next build will work?

JamO


On 11/15/19 11:04 AM, Srinivas Rachakonda wrote:

Hi,

I am seeing a new error.

sent 8 bytes  received 87 bytes  190.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1650) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)
mv: cannot stat ‘local.conf_compute_10.30.170.65’: No such file or directory
Warning: Permanently added '10.30.170.79' (ECDSA) to the list of known hosts.
Connection to 10.30.170.79 closed.
tempest results not found in /opt/stack/tempest/.stestr/0
Robot results publisher started...
-Parsing output xml:
Failed!
hudson.AbortException: No files found in path /w/workspace/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-rocky-dcgw-magnesium with configured filemask: output.xml
	at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:78)
	at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:54)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3052)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to prd-centos7-robot-2c-8g-13687
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:957)
		at hudson.FilePath.act(FilePath.java:1072)
		at hudson.FilePath.act(FilePath.java:1061)
		at hudson.plugins.robot.RobotParser.parse(RobotParser.java:49)
		at hudson.plugins.robot.RobotPublisher.parse(RobotPublisher.java:224)
		at hudson.plugins.robot.RobotPublisher.perform(RobotPublisher.java:246)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:79)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1840)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
[PostBuildScript] - Executing post build scripts.
[netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-rocky-dcgw-magnesium] $ /bin/bash /tmp/jenkins198548001739126173.sh
Archiving csit artifacts

          
R.Srinivas
+91-9243478719



On Fri, 15 Nov 2019 at 22:19, Luis Gomez <ecelgp@...> wrote:
FYI I have already merged Stanislav patches that allow projects to opt out from JVM monitoring. This means your CSIT should be working back now.

Stanislav, netvirt project has filed this patch:


Meaning they are still interested in the framework, can you please test it along with the JVM framework? if it works we can re-enable JVM monitoring in the netvirt jobs.

BR/Luis




On Nov 15, 2019, at 1:55 AM, <srinivas.rachakonda@...> <srinivas.rachakonda@...> wrote:

Hi Faseela,
 
I have raised the below review:
 
 
Thanks,
Srinivas
+91-9243478719
 
From: netvirt-dev@... <netvirt-dev@...> On Behalf Of Faseela K
Sent: 15 November 2019 14:03
To: Luis Gomez <ecelgp@...>
Cc: odl netvirt dev <netvirt-dev@...>; Jamo Luhrsen <jluhrsen@...>; integration-dev@...
Subject: Re: [netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Srinivas,
   As you have raised NETVIRT-1640 to track the issue, you can add the exception to the CSIT whitelist, so that CSIT is unblocked.
   We can remove it from the list whenever issue is fixed.
Thanks,
Faseela
 
On Thu, 14 Nov 2019 at 23:39, Luis Gomez <ecelgp@...> wrote:
Also looping netvirt list, to see how they want to proceed here.


On Nov 14, 2019, at 10:06 AM, Luis Gomez via Lists.Opendaylight.Org <ecelgp=gmail.com@...> wrote:
 
Thanks Jamo, I guess if netvirt project is interested in the JVM framework they can patch as you suggest, otherwise we can just disable it for netvirt so we have less things to maintain in integration.
 
BR/Luis
 


On Nov 14, 2019, at 9:53 AM, Jamo Luhrsen <jluhrsen@...> wrote:
 

or you can add the exception to this list:


https://github.com/opendaylight/integration-test/blob/master/csit/libraries/netvirt/excepts.py


eventually, assuming magnesium starts to work with decanter, you will hopefully
get the jvm monitoring.

JamO

On 11/14/19 9:44 AM, Luis Gomez wrote:
The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring. 
 
 
BR/Luis
 
On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:
 
This is one possibility.
 
Can someone more experienced than me to see this recomendation about elasticsearch:
Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:
Client client = new TransportClient()
                .addTransportAddress(new InetSocketTransportAddress(
                        "143.79.236.xxx",
                        9300));
 
Maybe it is also our problem....
Stan
 
From: Srinivas Rachakonda <srinivas.rachakonda@...
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi, 
 
Can the changes be reverted back as many CSIT jobs are failing. 
 
 

From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Well  I was wrong that was the reason.
I tried to run the failing job without JVM support 
(patched with the new patch which I prepared:
Setting the parameter
ELASTICSEARCHATTRIBUTE=disabled
Will disable all JVM related stuff in job)
Here 
the job is without those failures
So I conclude that there is really some connection between JVM support and those failures.
 
I also compared these two outputs for the same job:
 
Currently I do not know what is the reason for faulures or how to fix it...
Stan
 
 
 
 
From: Luis Gomez <ecelgp@...
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.
 
On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:
 
Hi all
I tried to analyze the problem.
I think that it is due to the fact that patch
which removed the explicit mention of stuff related to JVM from scriptplan and testplan
was merged on Nov 12 6:53 PM
Was merged on Nov 11 9:19 PM
 
And this build 
was executed on Nov 12, 2019 11:15:47 AM)
 
We well see if the next build is going to be successfull
 
Stan
 
From: srinivas.rachakonda@... <srinivas.rachakonda@...
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi All,
 
Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.
 
 
 
Yesterday there was a commit in builder file: jjb/integration/common-functions.sh
 
<image001.png>
 
Could this be the cause of failure. 
Please kindly help.
 
Thanks,
Srinivas
+91-9243478719
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-
 
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
 
View/Reply Online (#13813): https://lists.opendaylight.org/g/integration-dev/message/13813
Mute This Topic: https://lists.opendaylight.org/mt/55406880/535209
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [jluhrsen@...]
-=-=-=-=-=-=-=-=-=-=-=-
 
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13815): https://lists.opendaylight.org/g/integration-dev/message/13815
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8403): https://lists.opendaylight.org/g/netvirt-dev/message/8403
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1993029
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [k.faseela@...]
-=-=-=-=-=-=-=-=-=-=-=-

 
-- 
Faseela.K

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8406): https://lists.opendaylight.org/g/netvirt-dev/message/8406
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1965958
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [srinivas.rachakonda@...]
-=-=-=-=-=-=-=-=-=-=-=-


Stanislav Chlebec <Stanislav.Chlebec@...>
 

Hi all

I will check...

One error was probably in my patch – but I do not know it this is a culprit:

https://git.opendaylight.org/gerrit/c/releng/builder/+/85808/2/jjb/integration/common-functions.sh

I apologize for that.

 

Stanislav

 

From: Jamo Luhrsen <jluhrsen@...>
Sent: Friday, November 15, 2019 8:11 PM
To: Srinivas Rachakonda <srinivas.rachakonda@...>; Luis Gomez <ecelgp@...>
Cc: Srinivas Rachakonda <srinivas.rachakonda@...>; Stanislav Chlebec <Stanislav.Chlebec@...>; Faseela K <k.faseela@...>; odl netvirt dev <netvirt-dev@...>; integration-dev@...
Subject: Re: [netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Looks like stacking failed. Compute node failed because it couldn't
clone the neutron repo:


https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-rocky-dcgw-magnesium/69/control_1/stack.log.gz


Hopefully an intermittent problem in upstream neurton infrastructure.
maybe next build will work?

JamO

On 11/15/19 11:04 AM, Srinivas Rachakonda wrote:

Hi,

 

I am seeing a new error.

 

sent 8 bytes  received 87 bytes  190.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1650) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)
mv: cannot stat ‘local.conf_compute_10.30.170.65’: No such file or directory
Warning: Permanently added '10.30.170.79' (ECDSA) to the list of known hosts.
Connection to 10.30.170.79 closed.
tempest results not found in /opt/stack/tempest/.stestr/0
Robot results publisher started...
-Parsing output xml:
Failed!
hudson.AbortException: No files found in path /w/workspace/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-rocky-dcgw-magnesium with configured filemask: output.xml
  at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:78)
  at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:54)
  at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3052)
  at hudson.remoting.UserRequest.perform(UserRequest.java:212)
  at hudson.remoting.UserRequest.perform(UserRequest.java:54)
  at hudson.remoting.Request$2.run(Request.java:369)
  at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:748)
  Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to prd-centos7-robot-2c-8g-13687
         at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
         at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
         at hudson.remoting.Channel.call(Channel.java:957)
         at hudson.FilePath.act(FilePath.java:1072)
         at hudson.FilePath.act(FilePath.java:1061)
         at hudson.plugins.robot.RobotParser.parse(RobotParser.java:49)
         at hudson.plugins.robot.RobotPublisher.parse(RobotPublisher.java:224)
         at hudson.plugins.robot.RobotPublisher.perform(RobotPublisher.java:246)
         at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:79)
         at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
         at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
         at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
         at hudson.model.Build$BuildExecution.post2(Build.java:186)
         at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
         at hudson.model.Run.execute(Run.java:1840)
         at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
         at hudson.model.ResourceController.execute(ResourceController.java:97)
         at hudson.model.Executor.run(Executor.java:429)
[PostBuildScript] - Executing post build scripts.
[netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-rocky-dcgw-magnesium] $ /bin/bash /tmp/jenkins198548001739126173.sh
Archiving csit artifacts
 

R.Srinivas

+91-9243478719

 

 

 

On Fri, 15 Nov 2019 at 22:19, Luis Gomez <ecelgp@...> wrote:

FYI I have already merged Stanislav patches that allow projects to opt out from JVM monitoring. This means your CSIT should be working back now.

 

Stanislav, netvirt project has filed this patch:

 

 

Meaning they are still interested in the framework, can you please test it along with the JVM framework? if it works we can re-enable JVM monitoring in the netvirt jobs.

 

BR/Luis

 

 

 



On Nov 15, 2019, at 1:55 AM, <srinivas.rachakonda@...> <srinivas.rachakonda@...> wrote:

 

Hi Faseela,

 

I have raised the below review:

 

 

Thanks,

Srinivas

+91-9243478719

 

From: netvirt-dev@... <netvirt-dev@...On Behalf Of Faseela K
Sent: 15 November 2019 14:03
To: Luis Gomez <ecelgp@...>
Cc: odl netvirt dev <netvirt-dev@...>; Jamo Luhrsen <jluhrsen@...>; integration-dev@...
Subject: Re: [netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Srinivas,

   As you have raised NETVIRT-1640 to track the issue, you can add the exception to the CSIT whitelist, so that CSIT is unblocked.

   We can remove it from the list whenever issue is fixed.

Thanks,

Faseela

 

On Thu, 14 Nov 2019 at 23:39, Luis Gomez <ecelgp@...> wrote:

Also looping netvirt list, to see how they want to proceed here.

 

On Nov 14, 2019, at 10:06 AM, Luis Gomez via Lists.Opendaylight.Org <ecelgp=gmail.com@...> wrote:

 

Thanks Jamo, I guess if netvirt project is interested in the JVM framework they can patch as you suggest, otherwise we can just disable it for netvirt so we have less things to maintain in integration.

 

BR/Luis

 

 

On Nov 14, 2019, at 9:53 AM, Jamo Luhrsen <jluhrsen@...> wrote:

 

or you can add the exception to this list:


https://github.com/opendaylight/integration-test/blob/master/csit/libraries/netvirt/excepts.py


eventually, assuming magnesium starts to work with decanter, you will hopefully
get the jvm monitoring.

JamO

On 11/14/19 9:44 AM, Luis Gomez wrote:

The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring. 

 

 

BR/Luis

 

On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

This is one possibility.

 

Can someone more experienced than me to see this recomendation about elasticsearch:

Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:

Client client = new TransportClient()

                .addTransportAddress(new InetSocketTransportAddress(

                        "143.79.236.xxx",

                        9300));

 

Maybe it is also our problem....

Stan

 

From: Srinivas Rachakonda <srinivas.rachakonda@...
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi, 

 

Can the changes be reverted back as many CSIT jobs are failing. 

 

 


From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support 

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here 

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.

 

On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
 
View/Reply Online (#13813): https://lists.opendaylight.org/g/integration-dev/message/13813
Mute This Topic: https://lists.opendaylight.org/mt/55406880/535209
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [jluhrsen@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13815): https://lists.opendaylight.org/g/integration-dev/message/13815
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8403): https://lists.opendaylight.org/g/netvirt-dev/message/8403
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1993029
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [k.faseela@...]
-=-=-=-=-=-=-=-=-=-=-=-


 

-- 

Faseela.K

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8406): https://lists.opendaylight.org/g/netvirt-dev/message/8406
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1965958
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [srinivas.rachakonda@...]
-=-=-=-=-=-=-=-=-=-=-=-

 


Stanislav Chlebec <Stanislav.Chlebec@...>
 

Hi Luis
I will test it.

Stanislav

 

From: Luis Gomez <ecelgp@...>
Sent: Friday, November 15, 2019 5:49 PM
To: Srinivas Rachakonda <srinivas.rachakonda@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: Faseela K <k.faseela@...>; odl netvirt dev <netvirt-dev@...>; Jamo Luhrsen <jluhrsen@...>; integration-dev@...
Subject: Re: [netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

FYI I have already merged Stanislav patches that allow projects to opt out from JVM monitoring. This means your CSIT should be working back now.

 

Stanislav, netvirt project has filed this patch:

 

 

Meaning they are still interested in the framework, can you please test it along with the JVM framework? if it works we can re-enable JVM monitoring in the netvirt jobs.

 

BR/Luis

 

 

 



On Nov 15, 2019, at 1:55 AM, <srinivas.rachakonda@...> <srinivas.rachakonda@...> wrote:

 

Hi Faseela,

 

I have raised the below review:

 

 

Thanks,

Srinivas

+91-9243478719

 

From: netvirt-dev@... <netvirt-dev@...> On Behalf Of Faseela K
Sent: 15 November 2019 14:03
To: Luis Gomez <ecelgp@...>
Cc: odl netvirt dev <netvirt-dev@...>; Jamo Luhrsen <jluhrsen@...>; integration-dev@...
Subject: Re: [netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Srinivas,

   As you have raised NETVIRT-1640 to track the issue, you can add the exception to the CSIT whitelist, so that CSIT is unblocked.

   We can remove it from the list whenever issue is fixed.

Thanks,

Faseela

 

On Thu, 14 Nov 2019 at 23:39, Luis Gomez <ecelgp@...> wrote:

Also looping netvirt list, to see how they want to proceed here.




On Nov 14, 2019, at 10:06 AM, Luis Gomez via Lists.Opendaylight.Org <ecelgp=gmail.com@...> wrote:

 

Thanks Jamo, I guess if netvirt project is interested in the JVM framework they can patch as you suggest, otherwise we can just disable it for netvirt so we have less things to maintain in integration.

 

BR/Luis

 




On Nov 14, 2019, at 9:53 AM, Jamo Luhrsen <jluhrsen@...> wrote:

 

or you can add the exception to this list:


https://github.com/opendaylight/integration-test/blob/master/csit/libraries/netvirt/excepts.py


eventually, assuming magnesium starts to work with decanter, you will hopefully
get the jvm monitoring.

JamO


On 11/14/19 9:44 AM, Luis Gomez wrote:

The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring. 

 

 

BR/Luis

 

On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

This is one possibility.

 

Can someone more experienced than me to see this recomendation about elasticsearch:

Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:

Client client = new TransportClient()

                .addTransportAddress(new InetSocketTransportAddress(

                        "143.79.236.xxx",

                        9300));

 

Maybe it is also our problem....

Stan

 

From: Srinivas Rachakonda <srinivas.rachakonda@...
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi, 

 

Can the changes be reverted back as many CSIT jobs are failing. 

 

 


From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support 

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here 

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.

 

On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
 
View/Reply Online (#13813): https://lists.opendaylight.org/g/integration-dev/message/13813
Mute This Topic: https://lists.opendaylight.org/mt/55406880/535209
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [jluhrsen@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13815): https://lists.opendaylight.org/g/integration-dev/message/13815
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8403): https://lists.opendaylight.org/g/netvirt-dev/message/8403
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1993029
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [k.faseela@...]
-=-=-=-=-=-=-=-=-=-=-=-


 

-- 

Faseela.K

 


Stanislav Chlebec <Stanislav.Chlebec@...>
 

Well I seems that Jamo was right – probably infrastructure.

S.

 

From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Monday, November 18, 2019 8:37 AM
To: Jamo Luhrsen <jluhrsen@...>; Srinivas Rachakonda <srinivas.rachakonda@...>; Luis Gomez <ecelgp@...>
Cc: Srinivas Rachakonda <srinivas.rachakonda@...>; Faseela K <k.faseela@...>; odl netvirt dev <netvirt-dev@...>; integration-dev@...
Subject: Re: [netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi all

I will check...

One error was probably in my patch – but I do not know it this is a culprit:

https://git.opendaylight.org/gerrit/c/releng/builder/+/85808/2/jjb/integration/common-functions.sh

I apologize for that.

 

Stanislav

 

From: Jamo Luhrsen <jluhrsen@...>
Sent: Friday, November 15, 2019 8:11 PM
To: Srinivas Rachakonda <srinivas.rachakonda@...>; Luis Gomez <ecelgp@...>
Cc: Srinivas Rachakonda <srinivas.rachakonda@...>; Stanislav Chlebec <Stanislav.Chlebec@...>; Faseela K <k.faseela@...>; odl netvirt dev <netvirt-dev@...>; integration-dev@...
Subject: Re: [netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Looks like stacking failed. Compute node failed because it couldn't
clone the neutron repo:


https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-rocky-dcgw-magnesium/69/control_1/stack.log.gz


Hopefully an intermittent problem in upstream neurton infrastructure.
maybe next build will work?

JamO

On 11/15/19 11:04 AM, Srinivas Rachakonda wrote:

Hi,

 

I am seeing a new error.

 

sent 8 bytes  received 87 bytes  190.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1650) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)
mv: cannot stat ‘local.conf_compute_10.30.170.65’: No such file or directory
Warning: Permanently added '10.30.170.79' (ECDSA) to the list of known hosts.
Connection to 10.30.170.79 closed.
tempest results not found in /opt/stack/tempest/.stestr/0
Robot results publisher started...
-Parsing output xml:
Failed!
hudson.AbortException: No files found in path /w/workspace/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-rocky-dcgw-magnesium with configured filemask: output.xml
  at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:78)
  at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:54)
  at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3052)
  at hudson.remoting.UserRequest.perform(UserRequest.java:212)
  at hudson.remoting.UserRequest.perform(UserRequest.java:54)
  at hudson.remoting.Request$2.run(Request.java:369)
  at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:748)
  Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to prd-centos7-robot-2c-8g-13687
         at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
         at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
         at hudson.remoting.Channel.call(Channel.java:957)
         at hudson.FilePath.act(FilePath.java:1072)
         at hudson.FilePath.act(FilePath.java:1061)
         at hudson.plugins.robot.RobotParser.parse(RobotParser.java:49)
         at hudson.plugins.robot.RobotPublisher.parse(RobotPublisher.java:224)
         at hudson.plugins.robot.RobotPublisher.perform(RobotPublisher.java:246)
         at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:79)
         at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
         at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
         at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
         at hudson.model.Build$BuildExecution.post2(Build.java:186)
         at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
         at hudson.model.Run.execute(Run.java:1840)
         at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
         at hudson.model.ResourceController.execute(ResourceController.java:97)
         at hudson.model.Executor.run(Executor.java:429)
[PostBuildScript] - Executing post build scripts.
[netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-rocky-dcgw-magnesium] $ /bin/bash /tmp/jenkins198548001739126173.sh
Archiving csit artifacts
 

R.Srinivas

+91-9243478719

 

 

 

On Fri, 15 Nov 2019 at 22:19, Luis Gomez <ecelgp@...> wrote:

FYI I have already merged Stanislav patches that allow projects to opt out from JVM monitoring. This means your CSIT should be working back now.

 

Stanislav, netvirt project has filed this patch:

 

 

Meaning they are still interested in the framework, can you please test it along with the JVM framework? if it works we can re-enable JVM monitoring in the netvirt jobs.

 

BR/Luis

 

 

 

 

On Nov 15, 2019, at 1:55 AM, <srinivas.rachakonda@...> <srinivas.rachakonda@...> wrote:

 

Hi Faseela,

 

I have raised the below review:

 

 

Thanks,

Srinivas

+91-9243478719

 

From: netvirt-dev@... <netvirt-dev@...On Behalf Of Faseela K
Sent: 15 November 2019 14:03
To: Luis Gomez <ecelgp@...>
Cc: odl netvirt dev <netvirt-dev@...>; Jamo Luhrsen <jluhrsen@...>; integration-dev@...
Subject: Re: [netvirt-dev] [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Srinivas,

   As you have raised NETVIRT-1640 to track the issue, you can add the exception to the CSIT whitelist, so that CSIT is unblocked.

   We can remove it from the list whenever issue is fixed.

Thanks,

Faseela

 

On Thu, 14 Nov 2019 at 23:39, Luis Gomez <ecelgp@...> wrote:

Also looping netvirt list, to see how they want to proceed here.

 

On Nov 14, 2019, at 10:06 AM, Luis Gomez via Lists.Opendaylight.Org <ecelgp=gmail.com@...> wrote:

 

Thanks Jamo, I guess if netvirt project is interested in the JVM framework they can patch as you suggest, otherwise we can just disable it for netvirt so we have less things to maintain in integration.

 

BR/Luis

 

 

On Nov 14, 2019, at 9:53 AM, Jamo Luhrsen <jluhrsen@...> wrote:

 

or you can add the exception to this list:


https://github.com/opendaylight/integration-test/blob/master/csit/libraries/netvirt/excepts.py


eventually, assuming magnesium starts to work with decanter, you will hopefully
get the jvm monitoring.

JamO

On 11/14/19 9:44 AM, Luis Gomez wrote:

The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring. 

 

 

BR/Luis

 

On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

This is one possibility.

 

Can someone more experienced than me to see this recomendation about elasticsearch:

Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:

Client client = new TransportClient()

                .addTransportAddress(new InetSocketTransportAddress(

                        "143.79.236.xxx",

                        9300));

 

Maybe it is also our problem....

Stan

 

From: Srinivas Rachakonda <srinivas.rachakonda@...
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi, 

 

Can the changes be reverted back as many CSIT jobs are failing. 

 

 


From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support 

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here 

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.

 

On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
 
View/Reply Online (#13813): https://lists.opendaylight.org/g/integration-dev/message/13813
Mute This Topic: https://lists.opendaylight.org/mt/55406880/535209
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [jluhrsen@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13815): https://lists.opendaylight.org/g/integration-dev/message/13815
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8403): https://lists.opendaylight.org/g/netvirt-dev/message/8403
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1993029
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [k.faseela@...]
-=-=-=-=-=-=-=-=-=-=-=-


 

-- 

Faseela.K

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#8406): https://lists.opendaylight.org/g/netvirt-dev/message/8406
Mute This Topic: https://lists.opendaylight.org/mt/57925083/1965958
Group Owner: netvirt-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/netvirt-dev/unsub  [srinivas.rachakonda@...]
-=-=-=-=-=-=-=-=-=-=-=-