-- MikealRogers - 23 Jun 2006

Writing Chandler Automated Tests with CATS 0.2 Guide

This document describes how to write functional test cases for Chandler using the test tools and framework developed at OSAF. In addition it discusses some internals of the test framework and tools that may help in writing more advanced test cases.

This document assumes knowledge of the Python programming language and Chandler.

Dive right in - A sample test

Below is a simple test class that will create a new mail in Chandler.

import tools.cats.framework.ChandlerTestLib as QAUITestAppLib 1
from tools.cats.framework.ChandlerTestCase import ChandlerTestCase 2
from i18n.tests import uw 3

class !TestNewMail(ChandlerTestCase): 4

    def startTest(self): 5
        
        # creation
        mail = !QAUITestAppLib.UITestItem("MailMessage", self.logger) 6
        
        # action
        mail.SetAttr(displayName=uw("Invitation Mail"), toAddress="demo2@osafoundation.org", 
              body=uw("This is an email to invite you")) 7
        mail.SendMail() 
        
        # verification
        mail.Check_DetailView({"displayName":uw("Invitation Mail"),"toAddress":"demo2@osafoundation.org",
             "body":uw("This is an email to invite you")}) 8

• 1 Import new ChandlerTestLib framework. This is a real world example in which ChandlerTestLib was imported as QAUITestAppLib, this was done to make migration from cats 0.1 easier.
• 2 Import new ChandlerTestCase class, this class is intended to be inherited from by all test classes.
• 3 Since June 2006 we've has i18n support in Chandler and the test framework. When writing tests please try to keep internationalization in mind.
• 4 Each test class must have a unique name and inherit from ChandlerTestCase.
• 5 TestNewMail.startTest() will be the first (and currently only) method run by the test framework. All the operations in your test should be in this method.
• 6 This example shows a new mail item being created. Notice that after the item is created you can use and manipulate the object in your test class. Most method in ChandlerTestLib require the logger object to be passed in. You MUST pass self.logger to methods in ChandlerTestLib that require it, this logger object is instantiated and managed by the framework when the tests are run.
• 7 This example shows attributes being set on the mail item. These attributes will be set in the UI using the scripting library which manipulates the UI and sets these values.
• 8 This example shows a verification step. All tests should include verifications for any actions done in the UI. Never assume the UI or the item was actually manipulated because the method used to set the attributes didn't raise an error or return False.

So far ChandlerTestLib has done all the heavy lifting for you, but in many cases you will need to report pass/fail or time certain actions and events on your own outside of methods within ChandlerTestLib. In order to do that we first need to discuss some of the CATS 0.2 reporting structure.

CATS 0.2 uses a tiered encapsulation for reporting results:

Suite
  |
  Test
     |
     Action
          |
          Report (PASS/FAIL)
          Report (PASS/FAIL)
          |
     EndAction
     |
     Action
          |
          Report (PASS/FAIL)
          Report (PASS/FAIL)
          |
     EndAction
     |
  EndTest
   |
EndSuite

As you can see, Reports are called inside of actions, actions inside of tests, and tests inside of suites. The Test portion is everything run in the startTest() method. Notice that reports are just explicit PASS/FAIL notices, and are not timed, only the actions are timed.

A more advanced test.

import osaf.sharing.Sharing as Sharing
import osaf.sharing.ICalendar as ICalendar
import tools.cats.framework.ChandlerTestLib as QAUITestAppLib
from tools.cats.framework.ChandlerTestCase import ChandlerTestCase
import os, wx, sys
from osaf.pim import ListCollection
import osaf.pim.calendar.Calendar as Calendar

class TestExporting(ChandlerTestCase):

    def startTest(self):
        
        appView = self.app_ns.itsView
        path = os.path.join(os.getenv('CHANDLERHOME'),"tools/cats/DataFiles")
        filename = 'exportTest.ics'
        fullpath = os.path.join(path, filename)
        if os.path.exists(fullpath):
            os.remove(fullpath)
        
        #Upcast path to unicode since Sharing requires a unicode path
        path = unicode(path, sys.getfilesystemencoding())
        share = Sharing.OneTimeFileSystemShare(path, 'exportTest.ics', \ 
                ICalendar.ICalendarFormat, itsView=appView)
     
        self.logger.startAction("Export Test Calendar") <font style="font-family: Arial Black;" size=3 color="#ff0000">1</font>
        collection = ListCollection(itsView=appView)
        for event in Calendar.CalendarEvent.iterItems(appView):
            collection.add(event)
        share.contents = collection
        share.put()
        self.logger.report(True, name="share.put()") <font style="font-family: Arial Black;" size=3 color="#ff0000">2</font>
        self.logger.endAction(True) <font style="font-family: Arial Black;" size=3 color="#ff0000">3</font>

• 1 logger.startAction() begins the timer. First argument (string) is the unique name for the timing.
• 2 logger.report() logs pass/fail. First argument (boolean) is the result, option keyword arguments are name= and comments=. Although name is not required it is strongly encouraged. If report is called outside of an action encapsulation name becomes a required attribute as it is used to dynamically generate a new action encapsulation for the report. Calling logger.report() outside of action encapsulation, although technically supported, is strongly discouraged.
• 3 logger.endAction() ends the action timer. Passing a boolean to logger.endAction() is an easy way to call Pass/Fail at the end of the action. What it actually does is call logger.report(boolean) before ending the action.

Another very important note about this test is that there is no need for try: pass except: blocks to handle failures in python. If a test fails in python the traceback is captured, failure logged, and the state of the framework and test suite is cleaned up for you.

Depreciated logger methods and what to use instead

Depreciated logger.Start, logger.Stop

• for normal tests use logger.startAction, logger.endAction
• for performance tests use logger.startPerformanceAction, logger.endPerformanceAction

Depreciated logger.ReportPass, logger.ReportFail

• if you don't want to end the action
  • use logger.report(True/ False, comment='some string if you like')
• if you do want to end the action
  • use Logger.endAction(True/ False, comment='some string if you like'
  • Note when ending the action the comment string used in the startAction will be recorded in the log along with the pass/fail status and so a comment in the endAction is often not needed

Depreciated logger.Print

• use logger.comment

Other Depreciated methods

• logger.Report (with a capital R)
• logger.SetChecked
• logger.Close