Chandler Automated Test System (CATS)

Chandler Automated Test System (CATS) is a test framework built to write and execute python test cases for Chandler.

The framework uses CPIAScript for script execution but CATS has additional libraries built into it. The first library we built is QAUITestAppLib which encapsulates all the methods related to UI actions in Chandler. For instance, creating a new event, note, mail or task, filing in the title field, filing in the accounts dialog etc... It also has methods to verify data by going through the Chandler detail view as well as checking data in the repository. QAUITestAppLib removes the complexity from the test case so test development is easy. Some of the design points of the new framework are:

• Test script is isolated from application change by abstracting the methods in QAUITestAppLib.
• Framework provides concept of test suite and test scripts.
• Framework supports local test development and execution as well as test suite execution
• Framework supports test script level reporting as well as test suite summary report.

• Sample test script : CreateNoteTestScript
• Sample output from a testscript run: TestScriptOutput

We have also added a logging mechanism that logs the output of the test results as well as timing information for the various UI actions. The logger also summarises the results of multiple test runs at the end of the report.

Where CATS live

All the CATS libraries and scripts reside under: chandler/tools. The test scripts are under: chandler/tools/QATestScripts.

To add additional testscripts you would need to check out the chandler sources in read/write mode. For e.g

• svn co svn+ssh://svn.osafoundation.org/svn/chandler/trunk chandler

For additional information on checking out sources, please refer to the following page. In addition to this you will also need write privileges to the qa repository in svn. Please contact me for the above.

Running CATS

1. Define a CATSREPORTDIR environement variable to give the path where you want to save the report files.
   The default value is your Chandler directory.
   
   
2. First set the PARCELPATH environment variable to point to the cert parcel so you won't be prompted with the 'Do you want to trust this certificate' dialog. You can set the PARCELPATH to point to $CHANDLERHOME/tools/QATestScripts/DataFiles where a cert parcel is already checked in. Go to the Chandler directory and run cats with the following command. On Linux and Mac:

         cd $CHANDLERHOME
         release/RunChandler --create --stderr -f tools/QATestScripts/Functional/TestNewEvent.py
         or 
         release/RunChandler --create --stderr --scriptFile=tools/QATestScripts/Functional/TestNewEvent.py
         

   On Windows:

         cd $CHANDLERHOME
         ./release/RunChandler.bat --create --stderr -f `cygpath -w tools/QATestScripts/Performance/PerfStampEvent.py` 
         or 
         ./release/RunChandler.bat --create --stderr --scriptFile=`cygpath -w tools/QATestScripts/Performance/PerfStampEvent.py`
         

3. Alternately, you can also run the test scripts using the do_tests.sh script. This will over-ride your personal PARCELPATH environment variable and point it the certs parcel checked in. Currently we have a bug where we cannot run individual tests using do_tests.sh on windows (Bug 5029)

         cd $CHANDLERHOME
         tools/do_tests.sh -f  
         to run the entire functional test suite 
         or
         toos/do_tests.sh -p 
         to run the entire performance test suite
         or 
         tools/do_tests.sh -t !TestNewEvent.py 
         to run individual tests.
         

Performance Tests

A subset of performance tests as identified on the performance project page have been automated using CATS test scripts. The reside under: chandler/tools/QATestScripts/Performance. The data files used by these scripts are checked in under: qa/tools/QATestScripts/DataFiles

Documentation:

• Test scripts description