Meeting to discuss next generation of Chandler Desktop Test Automation System.

Attendees: John Anderson, Philippe Bossut, Katie Capps, Mikeal Rogers, Dan Steinicke, Heikki Toivonen, Bryan Stearns, Aparna Kadakia

Highlights

• John has made some progress with the record scripting test framework over the last few weeks. He has slowly been fixing problems reported by Dan. Bugs reported by Dan are all linked from here : http://chandlerproject.org/Projects/TestScriptRecordingAndPlayback
• John has been able to script modal dialogs but there are few bugs yet to be handled. There are some timing problems related to the multiple dialogs popping during an operation. For e.g. while subscribing to a collection if the timezone dialog opens up, hitting 'No' button on it has unexpected results. This could be fixed by waiting for the button name to change or recording delays. There are various workarounds but John hasn't picked one yet.
• The validation for each widget which has focus doesn't work properly yet. Additional validation will need to be added.
• Currently the scripting code generated by the recorder is highly terse and difficult to understand, as expressed by Dan. Katie suggested maybe we could record some of the standard motions and generate a human readable script that can then be used by developers and QA to gain better knowledge of how this works. Dan thought this was a great idea.
• Currently all the validation happens using "get value" method on the widget at run time. So what attributes are getting validated aren't explicitly in the scripts. Dan also requested that whatever fields get validated in the test cases need to be explicitly be dumped in the log file.
• Philippe asked whether resizing and DnD test cases work currently and John said resizing works but there are still some snags in DnD cases.
• Mikeal asked for documentation of whatever is already in place so the QA team can start looking at it. Also clearly documenting data structures that get turned into actions will be needed as part of it.
• There were some open questions around whether we should run the test on empty repository or prebuilt repository each time. No decision was made but both the options are available to choose from.
• Maintaining the script code is going to be hard for developers and QA. This requires lots of gnarly wx knowledge. For some time now till the framework has been properly developed the ownership of the framework will lie with John. John believes this framework, when completed, help us automate about 95% of the UI barring the edge cases. There still some areas that have problems but we will resort to safe workarounds in those cases.

Next Actions

• John will work on the bugs that hamper transfering the existing testcases over to the new framework. He will also work on documentation on the side.
• By next week, John might have something ready for the QA team to start helping out on. Two things that the QA team can help out on:
  • creating the human readable script from the recorded actions on Chandler
  • explicit output of what fields/attributes are getting validated in the log file
• Plan to meet next Thursday to follow up on the action items and get status update.

-- AparnaKadakia - 12 Sep 2007