Stamping Email Discussion

Questions

• What do Chandler users see when they receive plain ole email from standard email clients?
  • Who's in the From field?
  • Who's in the CC and BCC field?
  • Who's in the Send as field?
• When does Email end and when does Chandler communication begin?
• What happens if you remove email addresses from the addressing fields, but those people continue to edit and update the item?
• Are we going to preserve messages or do we just overwrite the previous version?

Scenarios Discussion

• Mitch/Esther share Mitch's calendar
• Esther created it in her Chandler, Mitch subscribes read-write
• The share is published on Cosmo
• Esther maintains a backup Mitch calendar in iCal and published it through .Mac
• Mitch subscribes to the iCal calendar read-only
• Both are using Apple Mail as their email client
• This scenario is a neat microcosm of 'end-user interoperability' issues:
  • Chandler to Chandler
  • Chandler to email clients
  • Chandler to iCal via email clients

Workflow: Creating, Sending and Receiving in Chandler

• • Mitch creates a new event on the shared Mitch calendar in Chandler - Dinner on Tuesday from 7PM - 10PM
  • Mitch wants to make sure Esther sees this
  • Mitch stamps the event to address it
    • From: mitch@osaf
    • To: esther@kei
    • Send as: mitch@osaf
  • Mitch wants Esther to know that...
    • He has added the event to the Chandler calendar; and that
    • Esther should add it to iCal
  • Mitch sends the addressed event to Esther
  • Mitch syncs his Chandler calendar and the addressed event goes up on Cosmo
  • Esther receives an email in Apple Mail with the event as an .ics attachment
    • Esther clicks on the .ics attachment to add the event to iCal. This didn't work and is logged as a bug.
    • Esther also wants to add the event to Chandler: First she tries to drag it into Chandler, then she tries to right click on the .ics attachment to open the attachment with Chandler and that actually works!
  • Esther then syncs the Mitch Calendar in Chandler. The version she pulls down from the server resolves itself with the version she added to Chandler.

• BKirsch - Everytime we mockup send/edit/update workflows we use an event as an example, is that intentional? Does send/edit/update only work with events?
• Mimi - No, this should work for all scenarios - Notes, Tasks, Events and any combination/permutation thereof.
• Reid - How is Chandler going to resolve the events that are received via email versus events received via sharing?
• Jeffrey - We need a UUID and a sequence number so we don't clobber changes to shared events when the emailed event shows up. Both the email and the shared item will have the same sequence number.
• fromMe-ness vs toMe-ness is not shared. Both Esther and Mitch see that the item is addressed in the same way:
  • From: Mitch
  • To: Esther
  • Sent by: Mitch
  • However, Mitch sees the item as Outbound, but Esther sees the item as Inbound.

How do we populate the email headers with the addressing fields?

• From email header = Send as addressing field
• To email header = To addressing field
• CC email header = CC addressing field + From addressing field, if the email address in the From addressing field is not already in the TO, CC or BCC fields.
• BCC email header = BCC addressing field

• Special Chandler header for item UUID + sequence number
• Special Chandler headers for any additional Chandler-specific attributes that are not already stored in the .ics attachment
  • Custom ticklers for Notes
  • Anytime-ness and Custom ticklers for Tasks and Events are stored in the .ics attachment

• BKirsch - Won't we end up in a situation where the Sender of an item always receives the item again in Chandler? How do we resolve that?
• Jeffrey - We can use the message ID to resolve the Sent item with the Received item. We also need to keep track of the sequence number so that the Received item doesn't overwrite the Sent item, if the user edited and iterated on the Sent item between the time they Sent the item and then Received it again via the Mail Service.
• The message ID needs to be persisted on the server and shared.

• Philippe - What if Esther receives the emailed item before she syncs the Mitch calendar in Chandler? We need to resolve the item she receives via Email with the one she already has in the shared collection, compare sequence numbers and essentially ignore the version she receives via Email if it is the same sequence number or a lower sequence number.

• Katie: The code that handles item matching...where does this live?
• Grant: This probably lives in the mail code - where we detect ics attachments and turn them into a Chandler events.
• Jeffrey: The mail service gets a message and figures out if there is a match with an existing item. Similarly, if the email arrives first and then the same item arrives via sharing, if the two versions of the item have the same sequence number, we ignore the shared version. If the shared version has a higher sequence number, we overwrite the version received via email.
• Katie: Is there duplicated logic?
• Bryan; Currently this is in the sharing code and the mail service calls it.
• Sheila: When do we bring Morgen into this conversation? How much of this should he have input on?
• Katie: Like everyone else, Morgen should understand whatever we decide to do here.
• Bkirsch - We should leverage the sharing layer.
• Next action Need a follow-up conversation with Morgen about this.
• Bryan - Question not about creating a new item it's about whether I ignore it or not.
• ?? Mimi: Not sure what this means
• Grant: There are actually three cases (i.e. it’s not just “create” vs “update”)
  1. Do nothing (it's a notification you sent, or something you already know about because the item is shared, and you already synced that change).
  2. Make a new item (it's an item you’ve never seen before).
  3. Sync an existing item.

Workflow: Receiving in Apple Mail

• Both Mitch and Esther receive Mitch's item as an email in Apple Mail
  • From: mitch@osaf
  • To: esther@osaf
  • CC: mitch@osaf
• Both Mitch and Esther see this email as an inbound email in Apple Mail
• In the message body, both Mitch and Esther see
  • An .ics attachment if the item sent was a task or event or both
  • The content of the item as text in the message body
    • Event-ness
    • Addressing fields
    • Title
    • Location
    • Date/time information: All-day-ness, Start date/time, End date/time, Timezone, Recurrence rule, Recurrence end date
    • Custom alarm dates
    • Notes

Receiving the item in Chandler

• In Chandler, we pull down the same message, we figure out if the item already exists in the recipien'ts repository, we compare sequence numbers and we discard whichever one has an earlier sequence number; or if they have the same sequence number, we discard whichever showed up later.
• When we display this in Chandler, we don't see the email headers in the addressing fields, instead, we see the Chandler addressing fields, stored in the same XML format we're using for sharing. So when Esther receives Mitch's email in Chandler, the Addressing fields she sees are:
  • From: mitch@osaf
  • To: esther@kei
  • CC: blank
  • BCC: blank
  • Sent by: mitch@osaf
• If there is XML in the message, we populate the Chandler addressing fields with whatever is defined in the XML
• If there is no XML in the message, we populate the Chandler addressing fields with whatever is defined in the email headers:
  • From email header = From Chandler addressing field and Send by Chandler addressing field
  • To email header = To Chandler addressing field
  • CC email header = CC Chandler addressing field

Getting tasks and event items into iCal and Chandler via .ics attachments received in Email clients

• Esther receives Mitch's event item in her Apple Mail
• Esther clicks on the .ics file to add it to iCal
• Esther drags the entire email into Chandler
• Esther drags the .ics file into Chandler
• Esther right clicks on the .ics file to open the attachment with Chandler
• Esther saves the .ics file and then drags or imports it into Chandler
• Esther saves the email and drags it into Chandler

• If Esther only manages to get the .ics file into Chandler, she will lose all of the metadata that is only stored in XML (e.g. Chandler addressing fields and Custom alarms)
• However, if Esther manages to the entire email message into Chandler, she will get all the metadata she would get if she synced her Mitch calendar with Cosmo and pulled down the version of the item that Mitch put up on Cosmo.

Subsequent Edits and Updates to Sent/Received items in Chandler

• Edits and Updates work pretty much the same as above.
• If Esther edits the item that Mitch sent her, the Chandler addressing fields will continue to be:
  • From: mitch@osaf
  • To: esther@kei
  • CC: blank
  • BCC: blank
• The only difference is that Sent by: mitch@osaf will turn into an Edited by: esther@kei pulldown where she can select from the various email accounts she's entered
• The item still has the same UUID and iCal UID
• However, it acquires a new message ID and sequence number as soon as Esther makes an edit.

Phasing Proposals that might save some development time

1: Simplify the byline

• Not distinguish updates from 1st time sent
• Not show edited state in the UI, ie. byline in the DV and the Comm status and Who columns in the Dashboard
• Only display the following 2 things in the byline
  • A Send as: pulldown for items that are in 'Draft state': Stamped to be Addressed, Not Sent, Sent but Edited
  • A Sent by xxx on xxx for items that have been 'Sent' and have not been subsequently Edited
• Engineers don't think that this will save us alot of time

2: Don't handler receiving email in Chandler at all

• It's kind of an all or nothing scenario
• Don't pull down email that in the INBOX
• Don't pull email email with special Chandler headers
• Don't pull down email from special Chandler IMAP folders
• Sheila: Why is this necessary?
• Mimi: Because it's weird if sometimes you receive email and other times you don't. Especially if the case where you don't receive email in Chandler is when the email was sent from another Chandler client.

3: Edit and Update only works for shared items

• Mimi: We don't ever want people to get themselves into a situation where they've Sent an item and then can't edit it again.

• Sheila: Eliminating pulling email into Chandler will impact people who just want to have a personal dashboard don't care about item-sharing, collaboration and editing/updating.

4: Disable sending items from Chandler and support out-of-band, low-fidelity email notifications instead

• This means Reply, Reply all and Forward as well
• Avoids complications around the edit and update workflows;
• While allowing us to continue to support getting email into Chandler via special IMAP folders as a way of populating your personal dashboard with information you receive via email; and
• Allows sharees to notify each other of changes without getting into the complexities of shipping Chandler items around and resolving edit/update conflicts
• Allow Chandler users to send task and event items as .ics attachments from their default email clients
• Grant: Launching the user's default email client from inside Chandler is tricky too.

5: Just track lastModifiedBy as a first step and don't distinguish between Edits and Sends.

• lastModifiedOn (date)
• lastModifiedBy (person)
• What is the attribute that was modified (or maybe cut this)
• Would need to track the sequence number as well. We can't rely on the clocks on different machines being in sync
  • We keep track of the sequence# from the last time the user synced with the server. We then increment the sequence number when the user edits their local copy of the item.
  • When the user syncs again, if the sequence# on the server and the sequence# locally have both incremented, then there is a conflict and someone has to win. (Yes, this is the same old conflict merge problem.)
• Bryan: Does having a single item exist in multiple collections exacerbate this problem?
• Jeffrey: Yes
• ?? Mimi: Why's that?
• Grant: If a single item is shared in multiple collections, then multiple sharees can change that item, leading to conflicts. In other words, if you share an item in two collections with sequence number 4, and two sharees make changes to each of the two copies, you have two different items with sequence number 5 when you next sync all your shares. Sharing over email is an extension of this problem, because each of the email’s recipients can change the item independently, and you only find out when you receive their updates.

• Mimi: How does this proposal impact the UI?
• Jeffrey: The "Send as" field with a drop down would always exist
• Mimi: We would need to populate that field differently if a different user comes along to edit and update the item. (e.g. Mitch's email accounts versus Esther's email accounts.)
• Mimi: Do we also display the lastModifiedBy information? (Maybe not.)

End-user Objectives

• #1 Reflect who last editied/changed something and when
• #2 Force a notification
• #3 Edit and Update items in-band; or item sharing via Email

• Get emails into Chandler to populate your Dashboard