performance meeting

Notes from a meeting w/Ted and Andi last week about performance and the repository. In particular, we agreed there were two kinds of effort to talk about:

• (A) Speed up development time (relatively quick fixes)
  • speed up unit tests, ideally by an order of magnitude
  • speed up application start time
    • from a cold start (first launch, creating a new repository)
    • from a hot start (with an existing repository)

• (B) Address overall repository quality
  • set targets for performance and scalability
  • allocate resources for meeting those targets
    • machines and environment for benchmarking
    • may or may not require hiring a perf engineer
    • development time will impact the rest of the schedule
  • when do we start spending time on this?

• We've agreed to look at (A) immediately because startup time and unit tests are impacting the development cycle. Next actions:
  • Ted will go ahead and implement the load-from-known-good-repository feature. Its not very costly and should help the unit tests and cold-start of the application.
  • Andi will spend some time investigating the use cases in (A), trying to identify any low hanging fruit. Andi will likely enlist Morgen's help to look at how LoadParcels uses the repository.
  • To really speed up the goals in (A), it may require us to start down the (B) path.

• Andi had originally planned on looking at (B) only after the .4 release. We now think it will be worthwhile to invest some effort on (B) in the .4 release, at least some initial planning/investigation time. This includes setting targets and getting resources lined up. Next actions:
  • Katie to set up performance plan site (for entire app).
  • Ted to fill out repository piece of performance plan.

follow up

meeting with Morgen, Andi and Ted:

• LoadParcels is ~75% of a cold application launch (11.7/15 sec)
• LoadParcels more reasonable for hot launch (1.5/5.8 sec)
• Morgen and Andi have identified a solution that is likely to cut LoadParcels time in half: use the repository store's api directly, instead of constructing each item as a python object first. We convinced ourselves that this "bulk loading" api may also have broader use. This api is independent of the dbxml implementation.
• Ted is making good progress with the load-known-good-repository, will have measurements after refactoring some of the test cases to make use of the feature wisely.

next actions:

• Andi and Morgen will work on the parcel loader rewrites (including any work needed to make the apis public). Morgen will start scoping it out -- we anticipate a two week hit to Morgen and Andi's schedule.
• Andi and Ted to sync up on repository schedule/priorities

-- KatieCappsParlante - 16 Jun 2004