Performance Meeting Notes - June 13, 2005

Attendees

• Katie, Sheila, Aparna

Decisions and Next Steps

• Capture weekly notes to make the meeting more official. Sheila will be responsible for this.
• Need to close on target hardware for 0.6.
  • Sheila will draft a summary email of the current situation and recommendations and send to the ops team.
  • We will request a discussion during the 06/15 ops meeting to close on plan.
• Sheila will update the performance use case page based on Aparna's feedback.
• Sheila will communicate to the dev list that the weekly meeting is occurring and where to find the notes so others are aware of the discussions.
• Aparna and Anthony will try to get a benchmark for sharing performance scenarios by the end of the week 06/17 so we can discuss with Andi prior to him leaving for Europe.
• Katie will draft a proposal for a tool to load test data from excel into Chandler.
• Katie will also work on a list of performance tests to be written that we need and we don't have yet.

Notes

Performance Scenarios

• Reviewed performance scenarios for 0.6 - ZeroPointSixPerformanceUseCases.
• Aparna had some comments and suggestions about the structure.
• We decided that instead of 3 scenarios we would have a simple list of use cases. Sheila will update this page.
• We should run all use cases for 100, 1000, 3000 event calendars.
• Anthony has already started with a 1000 event calendar.

Target Hardware

• We need to close on this. Ideally we would like what the consumer would have bought in 2003.
• We currently have a PC and Linux machine that are a bit too fast (2.8 GHz instead of 2) but close to our target. We could reduce the RAM to 512 to make it closer to what we want.
• We don't have a good solution for the Mac. We are using Kiluaea but the numbers are off (too slow). Katie is recommending that we get a Mac Mini - 1.25 GH G4 with extra memory (comes with 256, want to get 512) around 1GH was our target.
• Sheila will send an email to Mitch and the ops team with what we have now, our target and our recommendations. We will also bring this up for discussion at the ops meeting and close on the next steps.

Sharing Benchmarks

• We need to come up with some benchmarks numbers for the sharing scenarios.
• It would be good to do this soon since Andi is leaving for Europe and we can discuss with him before he goes so he has a clear priority when he is away.
• Aparna and Anthony will work on this.

Data Loading Tool

* need a tool (python loader) for loading data from excel into Chandler * what is the next step for getting this done? Katie will write up a proposal for this.

General

• We have a number of different performance tests happening. automated tests that Ted and Morgen has written. Identify a time during the day when we can run these (they take 40 min).
• Automated tests - (sort of part of the unit test framework)
  • unit tests
  • tests that tinderbox watches continually
  • tests you should run before check in
  • perf tests run once a day

• Anthony's tests
  • silk test on windows - some performance - checkpoint builds
  • stopwatch perf tests (adhoc - when we need to do assessments).

• There might be another class of tests needed?
  • more automated tests that are different?
  • need to have some space for tests that are automated tests -> think about whether or not there is a need for a set of tests that run weekly.
  • are there any of the stopwatch tests that can be automated?
    • eventually using cpia script.
    • do we need anything in the interim?

• katie - will make a list of performance tests to be written that we need and don't have yet.