System Test Flow


Luis Gomez <luis.gomez@...>
 

Hi all,

 

I would like to share with you what Carol and I discussed last week about the flow for System Test:

 

Every time a project code merges to Master (same trigger as for Integration jobs):

 

1) Jenkins should build release vehicles and perform run check (as part of the build) to verify OSGi bundles are successfully loaded. If the build and the run check are successful, new release artifacts will be uploaded to Nexus.

 

2) If the above is successful Jenkins shall call Robot framework which should:

 

-          Instruct Controller test VM (SUT) to download release artifacts from Nexus and run the controller

-          Start the system test on SUT

-          File a bug to Bugzilla in case system test case fails

-          Clear downloaded release artifacts in SUT (so we start fresh every time)

-          Report system test result (OK, NOK) back to Jenkins

 

3) In case of Robot reports NOK, Jenkins will:

 

-          Delete uploaded release artifacts in Nexus (not sure if this is possible)

-          Notify the project about the fail. Test Fail details can be consulted in Bugzilla.

 

The project fixes the issue and everything starts over….

 

Does this make sense to you as well?

 

BR/Luis

 

 

 


Moiz Raja (moraja) <moraja@...>
 

Luis,

I think the Sanity Test that I am writing fits the bill for No. 1. I hope someone else is not trying to do the same thing.

-Moiz


From: integration-dev-bounces@... [integration-dev-bounces@...] on behalf of Luis Gomez [luis.gomez@...]
Sent: Monday, October 07, 2013 12:07 PM
To: integration-dev@...; infrastructure@...
Subject: [integration-dev] System Test Flow

Hi all,

 

I would like to share with you what Carol and I discussed last week about the flow for System Test:

 

Every time a project code merges to Master (same trigger as for Integration jobs):

 

1) Jenkins should build release vehicles and perform run check (as part of the build) to verify OSGi bundles are successfully loaded. If the build and the run check are successful, new release artifacts will be uploaded to Nexus.

 

2) If the above is successful Jenkins shall call Robot framework which should:

 

-          Instruct Controller test VM (SUT) to download release artifacts from Nexus and run the controller

-          Start the system test on SUT

-          File a bug to Bugzilla in case system test case fails

-          Clear downloaded release artifacts in SUT (so we start fresh every time)

-          Report system test result (OK, NOK) back to Jenkins

 

3) In case of Robot reports NOK, Jenkins will:

 

-          Delete uploaded release artifacts in Nexus (not sure if this is possible)

-          Notify the project about the fail. Test Fail details can be consulted in Bugzilla.

 

The project fixes the issue and everything starts over….

 

Does this make sense to you as well?

 

BR/Luis

 

 

 


Luis Gomez <luis.gomez@...>
 

Right, the run test in 1) is the one you are writing. I just added it here after our conversation this morning ;)

 

 

From: Moiz Raja (moraja) [mailto:moraja@...]
Sent: Monday, October 07, 2013 12:10 PM
To: Luis Gomez; integration-dev@...; infrastructure@...
Subject: RE: System Test Flow

 

Luis,

 

I think the Sanity Test that I am writing fits the bill for No. 1. I hope someone else is not trying to do the same thing.

 

-Moiz


From: integration-dev-bounces@... [integration-dev-bounces@...] on behalf of Luis Gomez [luis.gomez@...]
Sent: Monday, October 07, 2013 12:07 PM
To: integration-dev@...; infrastructure@...
Subject: [integration-dev] System Test Flow

Hi all,

 

I would like to share with you what Carol and I discussed last week about the flow for System Test:

 

Every time a project code merges to Master (same trigger as for Integration jobs):

 

1) Jenkins should build release vehicles and perform run check (as part of the build) to verify OSGi bundles are successfully loaded. If the build and the run check are successful, new release artifacts will be uploaded to Nexus.

 

2) If the above is successful Jenkins shall call Robot framework which should:

 

-          Instruct Controller test VM (SUT) to download release artifacts from Nexus and run the controller

-          Start the system test on SUT

-          File a bug to Bugzilla in case system test case fails

-          Clear downloaded release artifacts in SUT (so we start fresh every time)

-          Report system test result (OK, NOK) back to Jenkins

 

3) In case of Robot reports NOK, Jenkins will:

 

-          Delete uploaded release artifacts in Nexus (not sure if this is possible)

-          Notify the project about the fail. Test Fail details can be consulted in Bugzilla.

 

The project fixes the issue and everything starts over….

 

Does this make sense to you as well?

 

BR/Luis