Meeting with Asa Dotzler from Mozilla.org

On Friday, Nov 19, Heikki and I visited Mozilla.org to talk to Asa Dotzler, their QA Lead/Project Manager to understand more about open source QA processes and the tools they use. We had a good 2 hour session with Asa. I have listed some key take-aways from the discussion below.

Mozilla.org currently enjoys a volunteer base of thousands of people and bug report submissions of the volume of hundreds of thousands from the community/volunteers after each significant release. It definitely had the advantage of a parent company viz Netscape, which was more widely known in the industry that helped gather momentum of the volunteer base initially. But they have done a remarkable job in keeping that volunteer base growing if not just steady. From where we are with Chandler 0.4. we still need to develop a good usable feature set that can be released as a dog food release for exciting the community. We are hoping to achieve that with 0.5 release.

Here are some good suggestions from Asa on how to work closely with the community to get their support for the long term:

1. One of the key things about working with volunteers is making them feel we value their contributions. Appreciating their contributions by rewarding them with small gifts like chandler T-shirts, Mugs, Thank you notes, putting their names on our website etc would go a long way in reflecting that. Also incorporating their feedback into the face/design and crediting them for their work would show we value that. (FYI, we do have a page ChandlerContributors that lists everybody who helped. -- DuckySherwood - 24 Nov 2004)

2. A good idea may be to put up a page under chandler home called quality and listing the names of the people who have contributed to Chandler, called Volunteer Stars. Also list the areas in which they have contributed. For e.g. Mr. XYZ helped verify 200 bugs

3. While we do want to give the volunteers the option to choose what they want to do or how they want to contribute to chandler, the different tasks that could potentially be assigned to volunteers based on their interest level are :

• filing new bugs
• adding patches
• casting votes on bugs
• verifying bugs
• triaging bugs (finding dups and marking them)
• testing the installer etc

4. What channels to use to get the word out to the community about the latest releases :

• IRC chat sessions
• Bug days/Collaborative QA sessions on IRC
• Intense blogging about Chandler once it is dog-food release
• Training days for volunteers on IRC to train them on how to use Bugzilla etc
• Use help from Mozilla.org members so they can spread the word around in their community chat sessions, blogs etc

5. It is highly important to train the volunteers to help them be more productive than counter-productive. For e.g. if somone has evinced interest in filing feature bugs then it is important they understand that they first need to search the existing bug reports to see if it already exists before filing a new one. There should be a process page to help the volunteer training. Hopefully once the system is smoothly running then the old volunteers can help train the new volunteers etc.

6. As a QA lead it is very important to keep an eye on the bugzilla activity. Doing a quick scan of the bugzilla reports every morning to catch if there are any severe errors/crashes that has been newly filed could be useful. Subscribing to Google news alert on Chandler or subscribing to news groups that talk about chandler also helps keep a pulse on the community and their feedback on the project.

7. Creating relationships between volunteers and developers. There should be a common understanding in the developer team to not discourage or bluntly turn down suggestions from any volunteers. Again, knowing how hard it is to get volunteers, we as a team should be very careful with our responses to them. QA lead sort of becomes the liason between the volunteers and developers if there is any communication that needs to happen between them.

8. Posting design questions on newsgroups and gathering their input and feeding it back to the design team can show that community input matters. It has helped Mozilla to run polls on mozillazine after every big release. We will need a similar mechanism in place to run polls once Chandler is more widely used in the community.

9. Announcing chandler's releases on major news groups, weblogs and IRC sessions after every big release can help generate publicity and get volunteer traction. Also getting volunteers to give input on release notes and asking the dedicated set of volunteers if they would be interested in downloading the release bits early can show they are important to our project.

10. For every significant release, publish a page with 'What's new in this release'. Alert them if the release bits are not functioning as desired so they don't waste their time downloading and testing the release. We have to remain cognizant of the fact that some of these volunteers may be on a dial-up line which may take hours to download the bits.

11. Once we release the dog food release, Asa has volunteered to make an appearance on our IRC session to contribute to the testing as well as spreading the word in the mozilla community about Chandler.

12. Asa also demonstrated a tool called TestRunner^? which runs on top of Bugzilla which is used to record test cases and their status (passed, failed, bug #). Assigned Bug:2234 to Jurgen to install TestRunner^? on top of Bugzilla.

-- AparnaKadakia - 22 Nov 2004