Hi Denghui,

 

Thanks for sharing your test plan. It sounds very close to what we discussed yesterday, let me summarize the ideas we have in mind for system test development:

 

1) System test using existing framework (Java):

 

You do not need robot framework, just maven and java. This is what we discussed with the Cisco person yesterday. In this case you put the system test java code in our repository, and then you run it every time you build it in OpenDaylight Jenkins or locally in your computer. The Cisco people have already Java artifacts to put NB API REST requests. Good thing of this solution is that all the Java environment is already there so you can start writing test cases from day 1. The only issue so far is that you need to have something on SB to simulate the devices. The plan for this is to have either a simulator running in the same system (collocated with the controller) or on a separate instance.

 

2) System test adding robot framework (Python or Java):

 

This is actually your plan. Andrew (ODL sysadmin) could install Robot on existing OpenDaylight Jenkins and testers can do the same with their local computers. Then Jenkins can be used to build a release vehicle and execute robot test cases. Same as 1) you need to have something on SB to simulate the devices either collocated with the controller or on a separate machine.

 

3) System test using a new test bed (Python or Java):

 

This is our original idea for system test. We use existing OpenDaylight Jenkins just to trigger system test on a separate test bed made of 3-5 instances: 1.Robot framework, 2. Controller, 3. Device simulation. With this infrastructure you do real blackbox testing as the controller is running on its own instance, also there is no impact on existing OpenDaylight Jenkins because Robot and tests run on a separate instance as well. Only issue with this solution is that you need more infrastructure to debug your test code, like 3-5 VMs.

 

Next week we will have meeting with IBM QA people so it might be we come up with more ideas.

 

For your questions: Yes to 1. and 2.

 

BR/Luis

 

 

 

From: huang denghui [mailto:huangdenghui@...]
Sent: Friday, October 18, 2013 7:50 AM
To: Baohua Yang; Liguangpeng (Roc, IPTechnologyResearchDept&HW)
Cc: Luis Gomez; Carol Sanders
Subject: Draft plan for Test Tools.

 

Hi folks,

    

    I have a draft plan  in my mind about how do i go ahead on test tools area, and share it here, if you guys have comments on it, that would be great. 

 

Step information	Progress	Problem
1. Setup test tools in my laptop, it is used to verify whatever test case I write before commit to repository. Tools include (jenkins, robot)	Done: Install Jenkins successfully, and create a job for integration project Which will do three things automatically, a. pull latest code from that integration repository. b. build the latest code. TODO: c. execute robot cases which wrote for this project. Install robot framework. Write a shell script to execute all wrote robot cases, and it will be invoked in Jenkins job.	One job created for controller project, And it can pull latest code from controller repository, but build it failed,BTW, I build it successfully not in Jenkins job.
2.After all test tools works, Then take one module Such as (SwitchManager), dig out its all Northbound API.	TODO
3.Write robot test cases for that module (SwitchManager).	TODO
4.Test on my local environment.	TODO
5.Commit to integration repository.	TODO
6.Update those test cases to Wiki, hopefully, it can be used to reference.	TODO
Also questions in my mind: 1. Since we have own integration repository, So, i think all test cases should put in integration repository, right? 2.From my learning, the first release of ODL, there will be three different distribution, base, serviceprovider and virtualization, since controller is common function for three distribution, From test cases point of view, test cases for controller will be used in three distributions, So there must be a super set of test cases for all NB API, finally, those test cases will divided into three group according to distribution.right? Request to Carol, From MoM of US group, i learn that you already setup test tools successfully on your laptop. Since all other member may will do the same thing just like you done in your laptop, So if you have any configuration (jenkins job configuration, any shell scripts or something like that) you done successfully in your laptop, if it is convenient, Could you share those to us? I think that can save lots of times for other guys.