1. Topics
  2. Netvirt CSIT jobs are failing with Exception "Elasticsearch".

Netvirt CSIT jobs are failing with Exception "Elasticsearch".

Srinivas <srinivas.rachakonda@...>
 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/118/robot-plugin/log_full.html.gz

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

https://github.com/opendaylight/releng-builder/commit/a88663a066fff020a39b6b21ebdb5d520977e553

 

 

Could this be the cause of failure.

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

?
smime.p7s

Stanislav Chlebec <Stanislav.Chlebec@...>
 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

https://git.opendaylight.org/gerrit/c/integration/test/+/85394

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

The patch https://git.opendaylight.org/gerrit/c/releng/builder/+/85398

Was merged on Nov 11 9:19 PM

 

And this build

https://jenkins.opendaylight.org/releng/job/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/118/

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...>
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/118/robot-plugin/log_full.html.gz

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

https://github.com/opendaylight/releng-builder/commit/a88663a066fff020a39b6b21ebdb5d520977e553

 

 

Could this be the cause of failure.

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 
Luis Gomez
 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.

On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

Hi all
I tried to analyze the problem.
I think that it is due to the fact that patch
which removed the explicit mention of stuff related to JVM from scriptplan and testplan
was merged on Nov 12 6:53 PM
Was merged on Nov 11 9:19 PM
 
And this build 
was executed on Nov 12, 2019 11:15:47 AM)
 
We well see if the next build is going to be successfull
 
Stan
 
From: srinivas.rachakonda@... <srinivas.rachakonda@...> 
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi All,
 
Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.
 
 
 
Yesterday there was a commit in builder file: jjb/integration/common-functions.sh
 
<image001.png>
 
Could this be the cause of failure. 
Please kindly help.
 
Thanks,
Srinivas
+91-9243478719
 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-
Stanislav Chlebec <Stanislav.Chlebec@...>
 

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here

https://jenkins.opendaylight.org/sandbox/job/stan-JVMdisable-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/1/

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

https://jenkins.opendaylight.org/sandbox/job/stan-fixed-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/ - with JVM support

https://jenkins.opendaylight.org/sandbox/job/stanislav-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/2/ - without JVM support

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...>
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...; integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.



On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...> 
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

 
Srinivas <srinivas.rachakonda@...>
 

Hi, 

Can the changes be reverted back as many CSIT jobs are failing. 


From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@...
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here

https://jenkins.opendaylight.org/sandbox/job/stan-JVMdisable-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/1/

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

https://jenkins.opendaylight.org/sandbox/job/stan-fixed-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/ - with JVM support

https://jenkins.opendaylight.org/sandbox/job/stanislav-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/2/ - without JVM support

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...>
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...; integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.



On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...> 
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

 
Stanislav Chlebec <Stanislav.Chlebec@...>
 

This is one possibility.

 

Can someone more experienced than me to see this recomendation about elasticsearch:

https://stackoverflow.com/questions/25912572/java-elasticsearch-none-of-the-configured-nodes-are-available

Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, the cluster.name as "elastictest" might be the problem. Try this:

Client client = new TransportClient()

                .addTransportAddress(new InetSocketTransportAddress(

                        "143.79.236.xxx",

                        9300));

 

Maybe it is also our problem....

Stan

 

From: Srinivas Rachakonda <srinivas.rachakonda@...>
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi, 

 

Can the changes be reverted back as many CSIT jobs are failing. 

 

 

From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@...
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here

https://jenkins.opendaylight.org/sandbox/job/stan-JVMdisable-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/1/

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

https://jenkins.opendaylight.org/sandbox/job/stan-fixed-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/ - with JVM support

https://jenkins.opendaylight.org/sandbox/job/stanislav-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/2/ - without JVM support

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...>
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...; integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.

 

On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...> 
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-

 

 
Luis Gomez
 

The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring.

On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

This is one possibility.
 
Can someone more experienced than me to see this recomendation about elasticsearch:
Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:
Client client = new TransportClient()
                .addTransportAddress(new InetSocketTransportAddress(
                        "143.79.236.xxx",
                        9300));
 
Maybe it is also our problem....
Stan
 
From: Srinivas Rachakonda <srinivas.rachakonda@...> 
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi, 
 
Can the changes be reverted back as many CSIT jobs are failing. 
 
 
From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support 

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here 

https://jenkins.opendaylight.org/sandbox/job/stan-JVMdisable-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/1/

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

https://jenkins.opendaylight.org/sandbox/job/stan-fixed-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/ - with JVM support

https://jenkins.opendaylight.org/sandbox/job/stanislav-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/2/ - without JVM support

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...> 
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...; integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.

 

On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...> 
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-
JamO Luhrsen
 

or you can add the exception to this list:


https://github.com/opendaylight/integration-test/blob/master/csit/libraries/netvirt/excepts.py


eventually, assuming magnesium starts to work with decanter, you will hopefully
get the jvm monitoring.

JamO
On 11/14/19 9:44 AM, Luis Gomez wrote:

The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring.

[1] https://git.opendaylight.org/gerrit/#/c/releng/builder/+/85756/

BR/Luis

On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

This is one possibility.
 
Can someone more experienced than me to see this recomendation about elasticsearch:
Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:
Client client = new TransportClient()
                .addTransportAddress(new InetSocketTransportAddress(
                        "143.79.236.xxx",
                        9300));
 
Maybe it is also our problem....
Stan
 
From: Srinivas Rachakonda <srinivas.rachakonda@...> 
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi, 
 
Can the changes be reverted back as many CSIT jobs are failing. 
 
 
From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support 

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here 

https://jenkins.opendaylight.org/sandbox/job/stan-JVMdisable-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/1/

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

https://jenkins.opendaylight.org/sandbox/job/stan-fixed-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/ - with JVM support

https://jenkins.opendaylight.org/sandbox/job/stanislav-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/2/ - without JVM support

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...> 
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...; integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.

 

On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...> 
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13813): https://lists.opendaylight.org/g/integration-dev/message/13813
Mute This Topic: https://lists.opendaylight.org/mt/55406880/535209
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [jluhrsen@...]
-=-=-=-=-=-=-=-=-=-=-=-

Luis Gomez
 

Thanks Jamo, I guess if netvirt project is interested in the JVM framework they can patch as you suggest, otherwise we can just disable it for netvirt so we have less things to maintain in integration.

BR/Luis


On Nov 14, 2019, at 9:53 AM, Jamo Luhrsen <jluhrsen@...> wrote:

or you can add the exception to this list:


https://github.com/opendaylight/integration-test/blob/master/csit/libraries/netvirt/excepts.py


eventually, assuming magnesium starts to work with decanter, you will hopefully
get the jvm monitoring.

JamO
On 11/14/19 9:44 AM, Luis Gomez wrote:
The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring.

[1] https://git.opendaylight.org/gerrit/#/c/releng/builder/+/85756/

BR/Luis

On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

This is one possibility.
 
Can someone more experienced than me to see this recomendation about elasticsearch:
Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:
Client client = new TransportClient()
                .addTransportAddress(new InetSocketTransportAddress(
                        "143.79.236.xxx",
                        9300));
 
Maybe it is also our problem....
Stan
 
From: Srinivas Rachakonda <srinivas.rachakonda@...> 
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi, 
 
Can the changes be reverted back as many CSIT jobs are failing. 
 
 
From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support 

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here 

https://jenkins.opendaylight.org/sandbox/job/stan-JVMdisable-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/1/

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

https://jenkins.opendaylight.org/sandbox/job/stan-fixed-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/ - with JVM support

https://jenkins.opendaylight.org/sandbox/job/stanislav-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/2/ - without JVM support

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...> 
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...; integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.

 

On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...> 
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13813): https://lists.opendaylight.org/g/integration-dev/message/13813
Mute This Topic: https://lists.opendaylight.org/mt/55406880/535209
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [jluhrsen@...]
-=-=-=-=-=-=-=-=-=-=-=-


Luis Gomez
 

Also looping netvirt list, to see how they want to proceed here.

On Nov 14, 2019, at 10:06 AM, Luis Gomez via Lists.Opendaylight.Org <ecelgp=gmail.com@...> wrote:

Thanks Jamo, I guess if netvirt project is interested in the JVM framework they can patch as you suggest, otherwise we can just disable it for netvirt so we have less things to maintain in integration.

BR/Luis


On Nov 14, 2019, at 9:53 AM, Jamo Luhrsen <jluhrsen@...> wrote:

or you can add the exception to this list:


https://github.com/opendaylight/integration-test/blob/master/csit/libraries/netvirt/excepts.py


eventually, assuming magnesium starts to work with decanter, you will hopefully
get the jvm monitoring.

JamO
On 11/14/19 9:44 AM, Luis Gomez wrote:
The real issue is apache decanter does not work in the latest Karaf version pulled in master. This generates an exception in the Karaf log and it seems netvirt tests are tracking and failing any test with karaf exceptions, so the solution is to merge this patch [1] and disable the framework in all netvirt jobs in master or in all netvirt jobs if netvirt project is not interested in JVM monitoring.

[1] https://git.opendaylight.org/gerrit/#/c/releng/builder/+/85756/

BR/Luis

On Nov 13, 2019, at 11:24 PM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

This is one possibility.
 
Can someone more experienced than me to see this recomendation about elasticsearch:
Elasticsearch settings are in $ES_HOME/config/elasticsearch.yml. There, if the cluster.name setting is commented out, it means ES would take just about any cluster name. So, in your code, thecluster.name as "elastictest" might be the problem. Try this:
Client client = new TransportClient()
                .addTransportAddress(new InetSocketTransportAddress(
                        "143.79.236.xxx",
                        9300));
 
Maybe it is also our problem....
Stan
 
From: Srinivas Rachakonda <srinivas.rachakonda@...> 
Sent: Thursday, November 14, 2019 8:20 AM
To: Luis Gomez <ecelgp@...>; Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: integration-dev@... <integration-dev@...>
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 
Hi, 
 
Can the changes be reverted back as many CSIT jobs are failing. 
 
 
From: Stanislav Chlebec <Stanislav.Chlebec@...>
Sent: Thursday 14 November 2019, 12:37 PM
To: Luis Gomez
Cc: srinivas.rachakonda@... ; integration-dev@... 
Subject: RE: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".
 

Well  I was wrong that was the reason.

I tried to run the failing job without JVM support 

(patched with the new patch which I prepared:

Setting the parameter

ELASTICSEARCHATTRIBUTE=disabled

Will disable all JVM related stuff in job)

Here 

https://jenkins.opendaylight.org/sandbox/job/stan-JVMdisable-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-gate-stateful-magnesium/1/

the job is without those failures

So I conclude that there is really some connection between JVM support and those failures.

 

I also compared these two outputs for the same job:

https://jenkins.opendaylight.org/sandbox/job/stan-fixed-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/1/ - with JVM support

https://jenkins.opendaylight.org/sandbox/job/stanislav-netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-dcgw-sodium/2/ - without JVM support

 

Currently I do not know what is the reason for faulures or how to fix it...

Stan

 

 

 

 

From: Luis Gomez <ecelgp@...> 
Sent: Wednesday, November 13, 2019 5:31 PM
To: Stanislav Chlebec <Stanislav.Chlebec@...>
Cc: srinivas.rachakonda@...; integration-dev@...
Subject: Re: [integration-dev] Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Correct there is a time gap between 2 patches that can cause issues in some CSITs. Please revert if problem persists.

 

On Nov 13, 2019, at 2:21 AM, Stanislav Chlebec <Stanislav.Chlebec@...> wrote:

 

Hi all

I tried to analyze the problem.

I think that it is due to the fact that patch

which removed the explicit mention of stuff related to JVM from scriptplan and testplan

was merged on Nov 12 6:53 PM

Was merged on Nov 11 9:19 PM

 

And this build 

was executed on Nov 12, 2019 11:15:47 AM)

 

We well see if the next build is going to be successfull

 

Stan

 

From: srinivas.rachakonda@... <srinivas.rachakonda@...> 
Sent: Wednesday, November 13, 2019 8:00 AM
To: integration-dev@...; Stanislav Chlebec <Stanislav.Chlebec@...>
Subject: Netvirt CSIT jobs are failing with Exception "Elasticsearch".

 

Hi All,

 

Test-netvirt-current CSIT jobs are failing due to issue/Exception with “Elasticsearch”.

 

 

 

Yesterday there was a commit in builder file: jjb/integration/common-functions.sh

 

<image001.png>

 

Could this be the cause of failure. 

Please kindly help.

 

Thanks,

Srinivas

+91-9243478719

 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13808): https://lists.opendaylight.org/g/integration-dev/message/13808
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13813): https://lists.opendaylight.org/g/integration-dev/message/13813
Mute This Topic: https://lists.opendaylight.org/mt/55406880/535209
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [jluhrsen@...]
-=-=-=-=-=-=-=-=-=-=-=-


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13815): https://lists.opendaylight.org/g/integration-dev/message/13815
Mute This Topic: https://lists.opendaylight.org/mt/55406880/1217165
Group Owner: integration-dev+owner@...
Unsubscribe: https://lists.opendaylight.org/g/integration-dev/unsub  [ecelgp@...]
-=-=-=-=-=-=-=-=-=-=-=-
1 - 10 of 10