• to be written in python
• must work the same on all these platform and python combinations
  • Windows + Chandler Python
  • Windows + Cygwin Python 2.4
  • Ubuntu + Chandler Python
  • Ubuntu + Python 2.4
  • PPC/Intel Mac + Chandler Python
  • PPC/Intel Mac + Python 2.3
• should be usable by all the different tinderboxes
  • full builds can use it to run unit tests
  • quick builds can use it to run unit and functional tests
  • performance boxes can use it to run unit, functional and performance tests
• should consolidate logging
  • logs from chandler, twisted, tests all in one log in chronological order
  • (bear) this may not be possible and it probably needs to be handled by Chandler
• might be nice if it would run functional tests on there own as well as in the suite (that is for each test in Functional directory start chandler with empty repository, run one test, close chandler).
• should log all commands passed to chandler so it is easy to know how to run a single with the same parameters do_tests uses
• should respond to both a SIG-KILL and a do_tests.stop semaphore file
• need to make sure any environment variables used or referenced in do_tests are in the new version

Options

• -m selects debug/ release mode; by default runs both debug and release if available

• -C continue tests even after test failure; by default stops on first error

• -u run all unit tests each in its own process
• -U run all unit tests in the same process
• -R run unit tests with a restored repository instead of creating new for each test

• -f run functional test suite
• -F run functional tests individually each in its own process

• -p run performance tests (each test is run 3 times and the median time is reported)
• -P profile performance tests

• -t run single test (note that can have some logic how to run specific tests)

• -T tbox output formatting; by default can output more user readable format
• -i ignore all environment variables (useful for tinderbox) - forces CHANDLERHOME to be current directory