Chandler Automated Test System (CATS)
Chandler Automated Test System (CATS) is a test framework built to write and execute python test cases for Chandler.
The framework uses CPIAScript for script execution but CATS has additional libraries built into it. The first library we built is
QAUITestAppLib
which encapsulates all the methods related to UI actions in Chandler. For instance, creating a new event, note, mail or task, filing in the title field, filing in the accounts dialog etc... It also has methods to verify data by going through the Chandler detail view as well as checking data in the repository.
QAUITestAppLib
removes the complexity from the test case so test development is easy. Some of the design points of the new framework are:
- Test script is isolated from application change by abstracting the methods in
QAUITestAppLib
.
- Framework provides concept of test suite and test scripts.
- Framework supports local test development and execution as well as test suite execution
- Framework supports test script level reporting as well as test suite summary report.
We have also added a logging mechanism that logs the output of the test results as well as timing information for the various UI actions. The logger also summarises the results of multiple test runs at the end of the report.
Where CATS live
All the CATS libraries and scripts reside under
chandler/tools
. The test scripts are under
chandler/tools/QATestScripts
.
To add additional testscripts you would need to check out the chandler sources in read/write mode. e.g.:
svn co svn+ssh://svn.osafoundation.org/svn/chandler/trunk chandler
For additional information on checking out sources, please refer to the following page. In addition to this you will also need write privileges to the qa repository in svn. Please contact me for the above.
Running CATS
- Define a
CATSREPORTDIR
environement variable to give the path where you want to save the report files.
The default value is your Chandler directory.
- Set the
PARCELPATH
environment variable to point to the cert parcel so you won't be prompted with the 'Do you want to trust this certificate' dialog. You can set the PARCELPATH
to point to $CHANDLERHOME/tools/QATestScripts/DataFiles
where a cert parcel is already checked in. Go to the Chandler directory and run CATS with the following command. On Linux and Mac:
cd $CHANDLERHOME
release/RunChandler --create --stderr -f tools/QATestScripts/Functional/TestNewEvent.py
or
release/RunChandler --create --stderr --scriptFile=tools/QATestScripts/Functional/TestNewEvent.py
On Windows:
cd $CHANDLERHOME
./release/RunChandler.bat --create --stderr -f `cygpath -w tools/QATestScripts/Performance/PerfStampEvent.py`
or
./release/RunChandler.bat --create --stderr --scriptFile=`cygpath -w tools/QATestScripts/Performance/PerfStampEvent.py`
- Alternately, you can also run the test scripts using the
do_tests.sh
script. This will over-ride your personal PARCELPATH
environment variable and adds OSAF CA certificates to the repository. It will also run the test using Chandler debug then release (so it runs the test twice).
cd $CHANDLERHOME
tools/do_tests.sh -f
to run the entire functional test suite or
tools/do_tests.sh -p
to run the entire performance test suite or
tools/do_tests.sh -t TestNewEvent.py
to run individual tests.
Performance Tests
A subset of performance tests as identified on the
performance project page have been automated using CATS test scripts. They reside under
chandler/tools/QATestScripts/Performance
. The data files used by these scripts are checked in under
qa/tools/QATestScripts/DataFiles
.
Documentation
Next Generation Ideas