[integration-dev] Making Sense of failing keystone test cases


Jamo Luhrsen <jluhrsen@...>
 

Hey Ryan,

(+Andy, Anil and Thanh)

I think you mean a CSIT job, not the maven verify job right? like this:

https://logs.opendaylight.org/releng/jenkins092/aaa-csit-1node-keystone-all-nitrogen/237/log.html.gz

I'm not sure on this one. /home/jenkins should be there, afaik. I dug back in that
jobs history, and found one where it was like that and working. Actually, I see that
it was fine in job 59 and broken for good in job 60. This was June 26th. I can't find
anything obvious in releng/builder or integration/test that gives a clue.

Andy, Thanh, Anil,

shouldn't this builder (CentOS 7 - docker - 20171016-0023) have /home/jenkins?

Thanks,
JamO

On 11/21/2017 11:31 AM, Ryan Goulding wrote:
Hi Integration gurus,

I am trying to figure out why the keystone related tests are failing.  I keep seeing this:

"02:42:15 ==============================================================================

02:42:15 aaa-keystone.txt
02:42:15 ==============================================================================
02:42:16 aaa-keystone.txt.Keystone Authn :: Test suite: Authentication Support for K...
02:42:16 ==============================================================================
03:44:10 Successful Authentication Including Domain :: *Test Case: Successf... | FAIL |
*03:44:10 Parent suite setup failed:*
*03:44:10 There was no directory matching '/home/jenkins'."*


Here is an example [0].  Any thoughts?
Thanks!

Ryan Goulding

[0] https://logs.opendaylight.org/releng/jenkins092/aaa-maven-verify-oxygen-mvn33-openjdk8/53/


_______________________________________________
integration-dev mailing list
integration-dev@...
https://lists.opendaylight.org/mailman/listinfo/integration-dev


Thanh Ha <thanh.ha@...>
 

Anil,

Any ideas why /home/jenkins is missing?

I thought our jenkins-scripts create this on bootup of the vm.

Regards,
Thanh

On Tue, Nov 21, 2017 at 5:44 PM, Jamo Luhrsen <jluhrsen@...> wrote:
Hey Ryan,

(+Andy, Anil and Thanh)

I think you mean a CSIT job, not the maven verify job right? like this:

https://logs.opendaylight.org/releng/jenkins092/aaa-csit-1node-keystone-all-nitrogen/237/log.html.gz

I'm not sure on this one. /home/jenkins should be there, afaik. I dug back in that
jobs history, and found one where it was like that and working. Actually, I see that
it was fine in job 59 and broken for good in job 60. This was June 26th. I can't find
anything obvious in releng/builder or integration/test that gives a clue.

Andy, Thanh, Anil,

shouldn't this builder (CentOS 7 - docker - 20171016-0023) have /home/jenkins?

Thanks,
JamO


On 11/21/2017 11:31 AM, Ryan Goulding wrote:
> Hi Integration gurus,
>
> I am trying to figure out why the keystone related tests are failing.  I keep seeing this:
>
> "02:42:15 ==============================================================================
>
> 02:42:15  aaa-keystone.txt
> 02:42:15  ==============================================================================
> 02:42:16  aaa-keystone.txt.Keystone Authn :: Test suite: Authentication Support for K...
> 02:42:16  ==============================================================================
> 03:44:10  Successful Authentication Including Domain :: *Test Case: Successf... | FAIL |
> *03:44:10 Parent suite setup failed:*
> *03:44:10 There was no directory matching '/home/jenkins'."*
>
>
> Here is an example [0].  Any thoughts?
> Thanks!
>
> Ryan Goulding
>
> [0] https://logs.opendaylight.org/releng/jenkins092/aaa-maven-verify-oxygen-mvn33-openjdk8/53/


Ryan Goulding <ryandgoulding@...>
 

Bumping this... trying to see how I can get these tests working again.  Is this the case, Anil?

Regards,

Ryan Goulding

On Wed, Nov 29, 2017 at 5:47 AM, Thanh Ha <thanh.ha@...> wrote:
Anil,

Any ideas why /home/jenkins is missing?

I thought our jenkins-scripts create this on bootup of the vm.

Regards,
Thanh


On Tue, Nov 21, 2017 at 5:44 PM, Jamo Luhrsen <jluhrsen@...> wrote:
Hey Ryan,

(+Andy, Anil and Thanh)

I think you mean a CSIT job, not the maven verify job right? like this:

https://logs.opendaylight.org/releng/jenkins092/aaa-csit-1node-keystone-all-nitrogen/237/log.html.gz

I'm not sure on this one. /home/jenkins should be there, afaik. I dug back in that
jobs history, and found one where it was like that and working. Actually, I see that
it was fine in job 59 and broken for good in job 60. This was June 26th. I can't find
anything obvious in releng/builder or integration/test that gives a clue.

Andy, Thanh, Anil,

shouldn't this builder (CentOS 7 - docker - 20171016-0023) have /home/jenkins?

Thanks,
JamO


On 11/21/2017 11:31 AM, Ryan Goulding wrote:
> Hi Integration gurus,
>
> I am trying to figure out why the keystone related tests are failing.  I keep seeing this:
>
> "02:42:15 ==============================================================================
>
> 02:42:15  aaa-keystone.txt
> 02:42:15  ==============================================================================
> 02:42:16  aaa-keystone.txt.Keystone Authn :: Test suite: Authentication Support for K...
> 02:42:16  ==============================================================================
> 03:44:10  Successful Authentication Including Domain :: *Test Case: Successf... | FAIL |
> *03:44:10 Parent suite setup failed:*
> *03:44:10 There was no directory matching '/home/jenkins'."*
>
>
> Here is an example [0].  Any thoughts?
> Thanks!
>
> Ryan Goulding
>
> [0] https://logs.opendaylight.org/releng/jenkins092/aaa-maven-verify-oxygen-mvn33-openjdk8/53/


Jamo Luhrsen <jluhrsen@...>
 

Hey Ryan,

you probably aren't on the integration-dev list, but I sent an email this morning
to Anil about this. Hopefully we can get a job in the sandbox running with this
problem so he can log in to the system live to see what's up. It's a matter
of us sync'ing up on our nearly opposite TZs

JamO

On 12/07/2017 11:04 AM, Ryan Goulding wrote:
Bumping this... trying to see how I can get these tests working again.  Is this the case, Anil?

Regards,

Ryan Goulding

On Wed, Nov 29, 2017 at 5:47 AM, Thanh Ha <thanh.ha@... <mailto:thanh.ha@...>> wrote:

Anil,

Any ideas why /home/jenkins is missing?

I thought our jenkins-scripts create this on bootup of the vm.

Regards,
Thanh


On Tue, Nov 21, 2017 at 5:44 PM, Jamo Luhrsen <jluhrsen@... <mailto:jluhrsen@...>> wrote:

Hey Ryan,

(+Andy, Anil and Thanh)

I think you mean a CSIT job, not the maven verify job right? like this:

https://logs.opendaylight.org/releng/jenkins092/aaa-csit-1node-keystone-all-nitrogen/237/log.html.gz
<https://logs.opendaylight.org/releng/jenkins092/aaa-csit-1node-keystone-all-nitrogen/237/log.html.gz>

I'm not sure on this one. /home/jenkins should be there, afaik. I dug back in that
jobs history, and found one where it was like that and working. Actually, I see that
it was fine in job 59 and broken for good in job 60. This was June 26th. I can't find
anything obvious in releng/builder or integration/test that gives a clue.

Andy, Thanh, Anil,

shouldn't this builder (CentOS 7 - docker - 20171016-0023) have /home/jenkins?

Thanks,
JamO


On 11/21/2017 11:31 AM, Ryan Goulding wrote:
> Hi Integration gurus,
>
> I am trying to figure out why the keystone related tests are failing.  I keep seeing this:
>
> "02:42:15 ==============================================================================
>
> 02:42:15  aaa-keystone.txt
> 02:42:15  ==============================================================================
> 02:42:16  aaa-keystone.txt.Keystone Authn :: Test suite: Authentication Support for K...
> 02:42:16  ==============================================================================
> 03:44:10  Successful Authentication Including Domain :: *Test Case: Successf... | FAIL |
> *03:44:10 Parent suite setup failed:*
> *03:44:10 There was no directory matching '/home/jenkins'."*
>
>
> Here is an example [0].  Any thoughts?
> Thanks!
>
> Ryan Goulding
>
> [0] https://logs.opendaylight.org/releng/jenkins092/aaa-maven-verify-oxygen-mvn33-openjdk8/53/
<https://logs.opendaylight.org/releng/jenkins092/aaa-maven-verify-oxygen-mvn33-openjdk8/53/>


Ryan Goulding <ryandgoulding@...>
 

Hi Jamo/Anil,

Following up and seeing if this ever happened. Please let me know if I should file a bug or join the integration meeting. Again, looking to either fix this test or disable it completely since it is clearly not working currently.

Thanks,
Ryan

On Dec 7, 2017, at 11:16 AM, Jamo Luhrsen <jluhrsen@...> wrote:

Hey Ryan,

you probably aren't on the integration-dev list, but I sent an email this morning
to Anil about this. Hopefully we can get a job in the sandbox running with this
problem so he can log in to the system live to see what's up. It's a matter
of us sync'ing up on our nearly opposite TZs

JamO

On 12/07/2017 11:04 AM, Ryan Goulding wrote:
Bumping this... trying to see how I can get these tests working again. Is this the case, Anil?

Regards,

Ryan Goulding

On Wed, Nov 29, 2017 at 5:47 AM, Thanh Ha <thanh.ha@... <mailto:thanh.ha@...>> wrote:

Anil,

Any ideas why /home/jenkins is missing?

I thought our jenkins-scripts create this on bootup of the vm.

Regards,
Thanh


On Tue, Nov 21, 2017 at 5:44 PM, Jamo Luhrsen <jluhrsen@... <mailto:jluhrsen@...>> wrote:

Hey Ryan,

(+Andy, Anil and Thanh)

I think you mean a CSIT job, not the maven verify job right? like this:

https://logs.opendaylight.org/releng/jenkins092/aaa-csit-1node-keystone-all-nitrogen/237/log.html.gz
<https://logs.opendaylight.org/releng/jenkins092/aaa-csit-1node-keystone-all-nitrogen/237/log.html.gz>

I'm not sure on this one. /home/jenkins should be there, afaik. I dug back in that
jobs history, and found one where it was like that and working. Actually, I see that
it was fine in job 59 and broken for good in job 60. This was June 26th. I can't find
anything obvious in releng/builder or integration/test that gives a clue.

Andy, Thanh, Anil,

shouldn't this builder (CentOS 7 - docker - 20171016-0023) have /home/jenkins?

Thanks,
JamO


On 11/21/2017 11:31 AM, Ryan Goulding wrote:
Hi Integration gurus,

I am trying to figure out why the keystone related tests are failing. I keep seeing this:

"02:42:15 ==============================================================================

02:42:15 aaa-keystone.txt
02:42:15 ==============================================================================
02:42:16 aaa-keystone.txt.Keystone Authn :: Test suite: Authentication Support for K...
02:42:16 ==============================================================================
03:44:10 Successful Authentication Including Domain :: *Test Case: Successf... | FAIL |
*03:44:10 Parent suite setup failed:*
*03:44:10 There was no directory matching '/home/jenkins'."*


Here is an example [0]. Any thoughts?
Thanks!

Ryan Goulding

[0] https://logs.opendaylight.org/releng/jenkins092/aaa-maven-verify-oxygen-mvn33-openjdk8/53/
<https://logs.opendaylight.org/releng/jenkins092/aaa-maven-verify-oxygen-mvn33-openjdk8/53/>


Jamo Luhrsen <jluhrsen@...>
 

https://lists.opendaylight.org/pipermail/integration-dev/2017-December/010651.html is
where we are discussing it.

the infra migration is still working out it's kinks so we haven't looked at the job
in the new infra yet.

However, if the job is not valuable to AAA and deleting it completely is on the table,
then just do that.

JamO

On 12/14/2017 03:56 AM, Ryan Goulding wrote:
Hi Jamo/Anil,

Following up and seeing if this ever happened. Please let me know if I should file a bug or join the integration meeting. Again, looking to either fix this test or disable it completely since it is clearly not working currently.

Thanks,
Ryan

Sent from my iPhone

On Dec 7, 2017, at 11:16 AM, Jamo Luhrsen <jluhrsen@...> wrote:

Hey Ryan,

you probably aren't on the integration-dev list, but I sent an email this morning
to Anil about this. Hopefully we can get a job in the sandbox running with this
problem so he can log in to the system live to see what's up. It's a matter
of us sync'ing up on our nearly opposite TZs

JamO

On 12/07/2017 11:04 AM, Ryan Goulding wrote:
Bumping this... trying to see how I can get these tests working again. Is this the case, Anil?

Regards,

Ryan Goulding

On Wed, Nov 29, 2017 at 5:47 AM, Thanh Ha <thanh.ha@... <mailto:thanh.ha@...>> wrote:

Anil,

Any ideas why /home/jenkins is missing?

I thought our jenkins-scripts create this on bootup of the vm.

Regards,
Thanh


On Tue, Nov 21, 2017 at 5:44 PM, Jamo Luhrsen <jluhrsen@... <mailto:jluhrsen@...>> wrote:

Hey Ryan,

(+Andy, Anil and Thanh)

I think you mean a CSIT job, not the maven verify job right? like this:

https://logs.opendaylight.org/releng/jenkins092/aaa-csit-1node-keystone-all-nitrogen/237/log.html.gz
<https://logs.opendaylight.org/releng/jenkins092/aaa-csit-1node-keystone-all-nitrogen/237/log.html.gz>

I'm not sure on this one. /home/jenkins should be there, afaik. I dug back in that
jobs history, and found one where it was like that and working. Actually, I see that
it was fine in job 59 and broken for good in job 60. This was June 26th. I can't find
anything obvious in releng/builder or integration/test that gives a clue.

Andy, Thanh, Anil,

shouldn't this builder (CentOS 7 - docker - 20171016-0023) have /home/jenkins?

Thanks,
JamO


On 11/21/2017 11:31 AM, Ryan Goulding wrote:
Hi Integration gurus,

I am trying to figure out why the keystone related tests are failing. I keep seeing this:

"02:42:15 ==============================================================================

02:42:15 aaa-keystone.txt
02:42:15 ==============================================================================
02:42:16 aaa-keystone.txt.Keystone Authn :: Test suite: Authentication Support for K...
02:42:16 ==============================================================================
03:44:10 Successful Authentication Including Domain :: *Test Case: Successf... | FAIL |
*03:44:10 Parent suite setup failed:*
*03:44:10 There was no directory matching '/home/jenkins'."*


Here is an example [0]. Any thoughts?
Thanks!

Ryan Goulding

[0] https://logs.opendaylight.org/releng/jenkins092/aaa-maven-verify-oxygen-mvn33-openjdk8/53/
<https://logs.opendaylight.org/releng/jenkins092/aaa-maven-verify-oxygen-mvn33-openjdk8/53/>


Jamo Luhrsen <jluhrsen@...>
 

This patch [0] should get your aaa-keystone job passing 100% now.

If I understand the test suite correctly, and assuming the feature to
use keystone for auth in ODL is valuable, this suite is a good one.

It'll be passing now and hopefully AAA can monitor it from time to time.

Thanks,
JamO

[0] https://git.opendaylight.org/gerrit/66687

On 12/14/2017 08:18 AM, Jamo Luhrsen wrote:
https://lists.opendaylight.org/pipermail/integration-dev/2017-December/010651.html is
where we are discussing it.

the infra migration is still working out it's kinks so we haven't looked at the job
in the new infra yet.

However, if the job is not valuable to AAA and deleting it completely is on the table,
then just do that.

JamO



On 12/14/2017 03:56 AM, Ryan Goulding wrote:
Hi Jamo/Anil,

Following up and seeing if this ever happened. Please let me know if I should file a bug or join the integration meeting. Again, looking to either fix this test or disable it completely since it is clearly not working currently.

Thanks,
Ryan

Sent from my iPhone

On Dec 7, 2017, at 11:16 AM, Jamo Luhrsen <jluhrsen@...> wrote:

Hey Ryan,

you probably aren't on the integration-dev list, but I sent an email this morning
to Anil about this. Hopefully we can get a job in the sandbox running with this
problem so he can log in to the system live to see what's up. It's a matter
of us sync'ing up on our nearly opposite TZs

JamO

On 12/07/2017 11:04 AM, Ryan Goulding wrote:
Bumping this... trying to see how I can get these tests working again. Is this the case, Anil?

Regards,

Ryan Goulding

On Wed, Nov 29, 2017 at 5:47 AM, Thanh Ha <thanh.ha@... <mailto:thanh.ha@...>> wrote:

Anil,

Any ideas why /home/jenkins is missing?

I thought our jenkins-scripts create this on bootup of the vm.

Regards,
Thanh


On Tue, Nov 21, 2017 at 5:44 PM, Jamo Luhrsen <jluhrsen@... <mailto:jluhrsen@...>> wrote:

Hey Ryan,

(+Andy, Anil and Thanh)

I think you mean a CSIT job, not the maven verify job right? like this:

https://logs.opendaylight.org/releng/jenkins092/aaa-csit-1node-keystone-all-nitrogen/237/log.html.gz
<https://logs.opendaylight.org/releng/jenkins092/aaa-csit-1node-keystone-all-nitrogen/237/log.html.gz>

I'm not sure on this one. /home/jenkins should be there, afaik. I dug back in that
jobs history, and found one where it was like that and working. Actually, I see that
it was fine in job 59 and broken for good in job 60. This was June 26th. I can't find
anything obvious in releng/builder or integration/test that gives a clue.

Andy, Thanh, Anil,

shouldn't this builder (CentOS 7 - docker - 20171016-0023) have /home/jenkins?

Thanks,
JamO


On 11/21/2017 11:31 AM, Ryan Goulding wrote:
Hi Integration gurus,

I am trying to figure out why the keystone related tests are failing. I keep seeing this:

"02:42:15 ==============================================================================

02:42:15 aaa-keystone.txt
02:42:15 ==============================================================================
02:42:16 aaa-keystone.txt.Keystone Authn :: Test suite: Authentication Support for K...
02:42:16 ==============================================================================
03:44:10 Successful Authentication Including Domain :: *Test Case: Successf... | FAIL |
*03:44:10 Parent suite setup failed:*
*03:44:10 There was no directory matching '/home/jenkins'."*


Here is an example [0]. Any thoughts?
Thanks!

Ryan Goulding

[0] https://logs.opendaylight.org/releng/jenkins092/aaa-maven-verify-oxygen-mvn33-openjdk8/53/
<https://logs.opendaylight.org/releng/jenkins092/aaa-maven-verify-oxygen-mvn33-openjdk8/53/>