Apps Team Meeting April 19th, 2007

This week's meeting is an "all hands" Chandler meeting. Since everyone is working on perf and that lots of cross Apps/Platform efforts are underway, it looked like a good idea to have a single common meeting instead of 2.

Agenda

• Question du Jour : (5 min) Any juicy/funny performance tweaking story?
• Performance work: (20 min) We're now 2 weeks in the Performance profiling period which will end on the 26th (see schedule^?). It's a good opportunity to look at the current results and the progress made and the work ahead. Grant did a great resume last week of what's going on. It's a good opportunity for everybody to update everybody else on the progress made and roadblocks found. We'll then try to make a list of the most promissing leads to orangify our current perf table.
• Other bugs: (10 min) Please do not completely neglect the other bugs! Though performance is a priority, fixing "blockers" has an even higher priority. So far, the Bug Council has been very reasonnable and has not raised a single bug to that level of emergency but it might be good to restate it may happen anytime. Also, in order to know if a bug is or not a blocker, QA may ask you sometimes a little more than just a quick look at the Bugzilla summary. Please be supportive of them: there's very few of them and there are a lot of scenarios to test. Also, right now, QA is scrambling to qualify Cosmo 0.6.1 and runs lots of test on EIM/MorseCode (which incidently allow us to start dogfooding for real). So your help to qualify those EIM related bugs (if any is assigned to you) is also very important.
• Issues and Announcements : (5 min)
  • Review period: update on the plan and open questions
  • Have something to announce? Issues to bring to the group? Please take the stage.

Minutes

Question du Jour

Any juicy/funny performance tweaking story?

• gbaillie: found a hotshot call in committed code!
• heikki: running without the indexer makes things waaay faster!

Performance work

• PPC is slow. What do we do? May be have lower acceptable numbers for PPC (Katie). We need to discuss that at the engineering meeting this afternoon.
• We are not testing perf with MorseCode yet. Morgen will switch the perf tests to MorseCode after this meeting. This may (or may not) improve the numbers for some core scenarios (sharing).
• We need to set numbers for Dump and Reload when that's done. Basically we think we need to make it twice as fast as it is today. Morgen hasn't worked on this yet (too busy with MorseCode and helping QA qualify Cosmo 0.6.1).
• Indexing
  • Andi stuck on digging through Esther's repo which incidentally is a good perf test case. Special kudos to Esther for her patience Found that BerkeleyDB isolation level used was set on too safe for our usage level. Then tried to compact the 6500 versions of the repo. Was able to finally fix that this morning. Thinking about making compaction run on a schedule (compaction can take several hours even on Intel Mac so fine line between too often and too little).
  • Stearns continued on indexing, improved the triaging perf from 10s to a little more than 1s!
• Hints mechanism : John worked on Alec's old code and simplified the design to be able to handle the events when they happened. Had then to fix the TBoxes failures. Realized that this factorization surfaced lots of calendar bugs. On the summary table, the hints we have are not precise enough to know if we need to refresh or not. Jeffrey to work on the calendar aspect next week. John and Andi will be looking into the table and events.
• Reid: looking into the new GraphicsContext instead of the DeviceContext, use native calls (Cocoa on Mac OS X, Cairo on Linux and GDI+ on Windows). 15% increase right of the bat simply switching to it. Robin helping there. Andi had used GraphicsContext for minical and this switch comes with new interesting features (like scaling). Be careful though that the bugs it shows are different and the workarounds also different. Bonus point: since Apple is discontinuing QuickDraw (won't be available for 64 bits apps), this work will help in the long run, beyond just improving the perfs (though the complete work will likely extend beyond Preview).
• Morgen: can someone help and look at EIM performance? bkirsch and heikkit to help out here. Andi mentions that importing ics is considerably slower, narrowed it to progress display. May be the import should import in a background view instead of the UI view so that the notification to the UI won't fire. Philippe to log a bug and assign to Jeffrey.

Status

Andi

• http://kumu.osafoundation.org/~vajda/logs.cgi

Brian (bkirsch)

• Added code to obfuscate export of mail records to EIM
• Submitted EuroPycon Sprint and Talk abstracts
• Helped in tracking down bugs related to Mail / UI including bug 8845
• Created two new Performance tests TestIncomingMail and TestOutgoingMail. The first measures performance metrics for converting various messages (EIMML, Mail, ICS) to Chandler Items. The second measures conversion of Chandler Items (Events, Tasks, Recurring Events) to RFC2882 mail messages. The later test reveals a 2+ second time to convert Items to text. This is too long.
• Will begin profiling EIM conversion of Items to Records
• Additional Profiling of Mail Observers toMe, fromMe, currentMeAddress, currentMeAddresses
• Research Hotels and Flights for EuroPycon

Bryan (stearns)

• Working on performance:
  • Investigation into the scrolling issue on Windows shows that scrolling is invalidating (& redrawing) all the visible cells, not just the ones that scrolled into view; this causes almost 5000 extra calls to the cell-drawing method. Trying to figure out why we suffer from this, but the wxDemo doesn't.
  • The triage perf test went from 12 seconds to just over a second (but varies by 40% routinely); haven't dug into what to change to get it faster.
• Participated in discussions about our performance monitoring - awaiting results.

Grant

• Progress
  • Added some doctests I had meant to add with a previous fix.
  • Committed a workaround we were seeing in the Flickr functional tests.
  • Defenestrated the osaf.pim.UserNotification class, and its uses in sharing; effectively these would be leaked in the old sharing world.
  • Summarized current state of Chandler Desktop performance efforts.
  • Got rid of the various “naïve” event indexes we have, in favour of querying for events a little differently when the user has timezones disabled in the UI.
• Plan
  • Getting rid of the effectiveStartTime computed attribute
  • Looking at proliferation of birefs in the detail view (i.e. having to change one biref per Block when committing after the UI changes).

Jeffrey

• Your status here...

John

• Made a pass through the BlockEvents to rationalize commitAfterDispatch
• Removed the reload parcels command along with its friends, since dump and reload perform an almost identical task and reload parcels was not reliable.
• Started to make a pass through the dialogs to make sure we have our commit story right. While I was at it, I removed unnecessary arguments for parent and id, removed unused dialogs, and replaced lots of our dialogs with the much simpler wx.MessageBox API. It turned out that this task was larger than anticipated, so I didn't finish the commit task and left in a few dialogs that could be refactored to eliminate about 80% of the existing code so I could focus on performance tasks.
• Updated the wing project files.
• Tried a bunch of changes to see if the occasional focus failures of the recorded tests scripts on tinderbox could be affected. I finally figured out that recording and playing back a mouse up event seemed to be responsible for the failures. On Mac and Linux, apparently mouse up events don't come at the same time as on Windows. So I stopped recording mouse up events and the tests seem to be passing.
• Reduced copying of blocks to improve performance.
• Spent a bunch of time exploring replacement of the partially implemented hints mechanism with a much simpler approach that is faster and more flexible. I'm close to having an implementation plan, which I expect to be able to implement and test in about a day.

• I spent some time on lingering problems resulting from my pass through the dialogs
• Finished refactoring the hints mechanism as part of my performance work. This exposed a bunch of long standing bugs in the calendar. I added workarounds for these bugs, but more work remains for the calendar to correctly save and use hints. I described this work in a task added to bugzilla, but didn't finish it.
• Rewrote the focus validation code used in recorded script, which was blocking QA adding new tests.
• Spent a bunch of time looking into how to use hints in the table and ran into a number of problem, most notably the hint for remove doesn't have enough information to be useful
• Briefly looked at the table scrolling performance problem, however I still don't understand the source of the problem.

Katie

• Progress
  • Bug:6553 (Offline mode for all)
  • Menu cleanup: Bug:8676, Bug:8012, Bug:6512, Bug:8639
  • Feb summaries: (Feb 5 - 11, Feb 12 - 18, Feb 19 - 25, Feb 26 - March 4)
  • March summaries: (March 5 - 11, March 12 - 18, March 19 - 25, March 26 - April 1)
  • Wiki gardening
  • Preview activities
  • chandler-user discussions
• Plan
  • get test data off the ground
  • blog post update about schedule
  • writeup about getting a few users before Preview

Morgen

• Dump/reload improvements and bug fixes
• Cosmo interop
• Added a "sharing error details" dialog and the underlying machinery to record the information that is displayed in it
• Obufscated dumps
• Improved sharing error messages
• Improved EIM support for adding/removing filters
• Large upload requests to Cosmo are now broken up into 100-item chunks
• Wrote some performance scripts for profiling reload, publish, subscribe, and sync

Philippe (pbossut)

• Perf Tweaking: monitored aspects of perf with team, used hotshot and contributed profiles to Bug#8758
• Preview bug duties: lots and lots and lots of bug triaging, review, tracking, etc...
• Intern: Usual activities (reviewed resumes, checked with candidates, sent around, discussed, etc...). Secured one new contract. Only 1 slot left to fill (IT intern for Jared).
• Reviews: Prepared peer reviewers list, reviewed forms

Phillip (PJE)

• Your status here...

Reid

• Working with new wx GraphicsContext class
• Found up to a 15% improvement (15% was for PerfLargeDataResizeCalendar)
• This is with just substituting the gradient code and changing the dc to a GCDC via dc = wx.GCDC(dc_in)
• There are benefits to be had, and with the GCDC class, the improvements can be incremental over time; no need to dive in all at once

Robin

• Reviewing calendar drawing code.
• Still working on Bug#7076 (Infinite redraws when editing in sidebar)
• Coordinating introductions, cvs access, and etc. for the wxPython and wxWidgets Summer of Code students.
• Provide wx support to Bryan and others as they are looking into performance issues related to wx.
• Worked on several bugs in wxPython 2.8/2.9 and applied a few patches from wxPython users.
• Spent some time dealing with web and mail server issues for the wxPython and alldunn sites after having some downtime problems. (Almost a whole a day without email!) Decided to go with a new hosting provider and will be migrating things over to it.
• Various community support activities via email and IRC.

-- PhilippeBossut - 19 Apr 2007