Item Collection Design Overview

Contents

• # Summary
• # Terminology
• # Use Cases
• # Structure
• # Workflows
• # Feature List
• # Open Issues
• # Notes
• # Comments

---

Summary

An attempt to communicate a coherent story about how users will manage sets of items in Canoga. Still working out terminology. Very much a work in progress.

---

Terminology

Warning: Can't find topic Documentation.TriageStatus

---

Use Cases

Background We've tried to avoid specific references to Chandler functionality in the use cases (the items in italics). For each use case, we've provided "real life" examples of PIM collections people make today. As you can see some of the examples live under seeminly mutually exclusive use cases (ie. the collection Project = Vacation to Hawaii can be viewed both as a carefully constructed explicit collection where all the member items have been added by hand OR it can be viewed as a pre-filtered set of dozens to hundreds of items that all share a common Project attribute). This demonstrates the amorphous nature of collections. What users expect of collections often changes over time (ie. Project=Vacation to Hawaii may start out as a carefully constructed explicit collection, but overtime grows into a behemoth pre-filtered set) which means that same collection will be expected to behave in very different ways and provide very different affordances depending on the every-changing needs and wants of the user. We could force users to choose between two kinds of collections, however even the most advanced users often don't know a priori how they might end up using a collection. This presents a unique challenge to us to provide a flexible interface that accomodates both shifting user needs and the every-present UI requirement for simplicity and clarity.

• To collect new material in one place
• Inbox
• Dashboard view (Processing bin)

• To keep track of conversations
• Email threads
• Invitation threads (notifications and email)
• [.4] Thrasks

• To put together an explicit list of items
• Task list: Grocery list
• Project = Vacation to Hawaii
• Calendars
• ie. [.4] My calendars: Home and Work
• ie. Other people's calendars: Spouse, family, working group, administrative assistant
• ie. Organizational calendars: Company, soccer team, performance series
• ie. FYI calendars: Holidays

• To facilitate searching
• Prefiltered set:
• ie. Mitch's "important email" archive
• ie. Calendars (see examples from above)
• ie. Project = Vacation to Hawaii
• ie. Auto-generated contact group: "Most frequently used contacts"
• ie. Contact group based on a "general category" such as "Work" or "Paris" which is unlikely to be used as an alias.
• ie. All notes about "Things to do in Napa"
• Ephemeral collections: Rule result set
• ie. Common searches: Who, timeframe, subject, status, project, category, text

• To create shortcut aliases
• Contacts
• ie. [.4] Groups based on organization: all@mycompany.com
• ie. [.4] Groups based on activity groups (ie.
• ie. Auto-generated: most frequently used contacts
• ie. All items associated with a contact

• To have a comprehensive archive of items related to a particular topic for future reference
• Mailing list
• [.4] Projects

• To catalogue and cross-reference collections for daily use
• For keeping track of large volumes of data (ie. Bear's Customer A, Food Vendor B, Import tool C example: to be elaborated upon, most similar to iTunes functionality)
• [OI^?] What is the OOTB^? set? Are there any OOTB?
• User-created from scratch

• To administrate Chandler functionality
• Notifications
• Sharing manager
• Recent changes
• Recently created items
• [.4] ALL items (as well as ALL items per Kind)
• Trash
• Junk mail

• 01_Collections_Types_Brains.gif:
  

• 02_Collections_UseCases_Bra.gif:
  

---

Structure

Background

We began with an examination of the existing collection types in current information management applications. (ie. PIMs such as Outlook, Apple PIM suite, iTunes, preliminary documentation of Longhorn) We came up with essentially two types of collections: Explicit and Rule-based collections and third (User-defined Program Actions) which is essentially a hybrid of the first two and fourth anomaly (Calendars).

For a more detailed discussion of these different kinds of collections, please refer to this table. (To be formatted as a wiki table)

• Collection_Types_001m033104.xls:

Most applications draw stark interaction distinctions between these different kinds of collections. The most important difference between Explicit and Rule-based collections is generally that user are not allowed to DnD items IN and OUT of rule-based collections. Instead to fine-tune rule-based collections, users must change attribute values on the items themselves or tweak the rule. Furthermore, users cannot perform DnD manipulations that violate the rule (ie. iTunes: Drag a song from January of 1970 into a Smart Playlist governed by the rule: All songs from the 1960s.) With User-defined Program Action Hybrid collections, the rules are a bit looser. Users can specify a rule to populate the collection, but they can further fine-tune the collection with direct DnD. However there are many more subtle differences between Rule-based collections and User-defined Program Action Hybrid collections that are discussed in more detail in the table. (ie. labeling effect, collection v. item primacy, collection-centric v. rule-centric)

We don't want to draw such stark interaction lines between collections. User-defined Program Action Hybrid collections are probably the best compromise between Explicit and Rule-based collections. They combine the flexibility of Explicit collections with the power-user automation affordances of Rule-based collections.

Oftentimes, users use rules to create a collection because it gets them 90% there. However, generally speaking, rules are too coarse grain to get exactly what you want, even for the most advanced users. After a certain point, the law of diminishing returns takes over and putting more effort into crafting more and more refined rules becomes a burden, a layer of indirection, a clumsy implement that the user wants to get rid of so that they can dig in with their bare hands to manipulate and fine-tune the collection with explicit DnD manipulations. You can also easily imagine the reverse workflow where users start out with an Explicit collection, DnDing items into the collection by hand and then add a program action rule to automatically add items in the future. (ie. Set up a calendar by hand and then create a program action rule to put all future invitations from my Mom on the Home calendar.)

However, User-defined Program Action Hybrid collections have quirky little behavioral tics that make it so that we can't get rid of Rule-based collections completely. (ie. Program Action rules do not "label" items that are DnDed into the collection. Program Action rules are not attached to the collection it populates. Program Actions move items between collections, rather than relying on a INCLUDE / EXCLUDE mental model, a relatively "antiquated" concept in the new Chandler paradigm where items can live in many locations.)

A survey of interaction rules for various types of collections

#	Collection type	Example	Persistence	DnD?	Rules	See all collections an ITEM is a member of	Change attributes on ITEM to add / remove from collection
What's out there
1	Explicit collections	iTunes playlist	Y	Y	N	N	N
2		Outlook folder	Y	Y	N	N	N
3	Rule-based collections	iTunes smart playlist	Y	N	Y	Sort of	Y
4		Outlook category view	Y	N	Y	Sort of	Y
5		Longhorn stacks	Y	Y	Y	Don't know	Y
6	Program Action collections	Outlook & Apple Mail mail filters	Y	Y	Y	N	N
Proposed Chandler collections
1	Ephemeral collections		N	N	Y	N	Y
2	Named collections		Y	Y	Y	Y	Y
3	Ad-hoc collections		Y	Y	N	Y	N (except for the CollectionName attribute

Both iTunes and Outlook draw very clear interaction lines between Explicit and Rule-based collections. Users must choose a priori which kind of collection they want with no affordances for turning one kind of collection into another kind of collection. UI affordances are strictly segregated with DnD reserved for Explicit collections. To change the membership of an item in rule-based collection, users can either fiddle with item attributes or refine the rule governing the collection. Longhorn stacks and Outlook and Apple Mail program action based mail filters allow for a little more flexibility.

Early on, Mitch decided that we should allow users to violate rule-based collections in the way they can violate program actions rules today. Based on a conversation with Andi, Mitch came up with the idea that we could allow users to DnD whatever they wanted into a rule-based collection largely because the repository would be able to keep track of all manipulations and restore history as needed. This further muddied the waters of an already complex issue. However, there were clearly compelling use cases to allow for this functionality. (ie. Mitch creates an iTunes smart playlist: All songs by Bob Marley. However, Mitch would like remove 1 song that he doesn't like. In order to do that in iTunes, he would have to change the Artist attribute on the song to something other than Bob Marley, which is both cumbersome AND inaccurate. Mitch should be able to drag the single undesired Bob Marley song out of the smart playlist and have the application remember and persist the exclusion.) As a result, we have made an attempt to come up with a relatively unified notion of collections. As you can see, Chandler's Named Collections accomodate all of the affordances in the table above in the hopes of providing users with a single, malleable solution to satisfy their collections needs.

Structure proposal: ATTEMPT AT UNIFICATION

Chandler will have 3 types of collections:

• Ephemeral collections are essentially what we think of us a query result sets. Ephemeral collections are temporary collections pulled together as the result of a rule. They possess most of the characteristics of rule-based collections. The most important shared characteristic is the equation of the rule with the collection, where the collection IS the rule rather than a container that is populated by the rule. The end-user will probably not think of Ephemeral collections as collections at all primarily because they will not appear in the "See also" section of member items' Detail views. Workflows for Ephemeral collections will live in a separate Search design document.
• Named collections Persistent collections that will contain a combination of the characteristics discussed above and in the table. We will talk about named collections more below.
• Ad-hoc collections Informal, easy to create, easy to access, low maintenance collections. See Ad-hoc collections workflow for more details.
• 3_types.gif:
  

Named collections

* named_collections.gif:

From Explicit collections
Users can create collections and DnD items IN and OUT at will.

From Rule-based collections
Users can create rules and save them as collections. OR conversely, users can create collections and add rules to "govern" them.

From Program Action Hybrid collections
Unlike "real" rule-based collections, where the collection IS the rule and the rule IS the collection (ie. Project = Foo, From = Mary are both the collection and the rule) named collections separate the notion of the rule from the collection. Instead, the user thinks of the rule as something that is populating the collection, where the collection is simply a container of items.

• rule_populates_container.gif:
  

From Rule-based collections
Unlike program action hybrid collections, named collections WILL force all DnD manipulations through the rule. As a result "label" attribute values defined in the rule will "label" all items added to the collection. (ie. If a collection is a populated by the rule Task status = Todo, all items that are DnDed into the collection will also be labeled "Todo".

• label_effect.gif:
  
• 002a_Inherent_v_labels.gif:
  

From Rule-based collections
Chandler will be a collection-centric world where rules belong to collections.

From Program Action Hybrid collections
Users can DnD items IN and OUT of the collection that essentially violate the rule and that in strict rule-based collections would simply be disallowed. For example, in the iTunes smart playlist: Songs from the 1960s, if a song has a year attribute value of 1963, the user cannot remove the song from the smart playlist without changing the year attribute value. We want users to be able to that. In the Chandler mental model of iTunes, when users create a smart playlist: Songs from the 1960s, we don't take them so literally. We assume that the original rule is only meant to get them 90% of what they want and that if they try to DnD items IN or OUT of the collection that violate the original rule, they're simply expressing a legitimate desire to fine tune the collection and they should be allowed to do so.

From Rule-based collections
Rule-based inclusions and exclusions from a collection are restorable.

Chandler bonus feature Individual DnD INs and OUTs are restorable as well.

How named collections differ from ad-hoc collections

• They don't open in place
• Ad-hoc collections can't have rules

Types of Named Collections

• Mailboxes
• Calendars
• Projects
• Spheres of life
• Contact groups
• Generic collections
• [OI^?] Where does subject fit in here?

Interaction paradigm: Old or New? [Seriously under construction]

* paradigm.gif:

What does move really mean in Chandler? In the old paradigm, where items could only live in one place, moving was the equivalent of taking an item from your IN tray and filing it away in a specific folder, in a specific drawer or a specific file cabinet. If you were especially studious, you might make a photo copy and file the photo copy in a different place for cross-referencing purposes.

In our new paradigms, items can live anywhere and everywhere. There isn't so much the notion of "moving" as there is the idea of "inclusion" and "exclusion" from particular collections. There are compelling advantages to thinking of "moving" items around in this way. For example, [insert Bear's example]

However, what we do lose is the sense of a life cycle, a workflow for items, how they are created and received when they are new, placed into an incubator as they are fleshed out, put into a processing bin, filed into task piles, dealt with and finally gotten rid of or filed away as reference. This is our natural way of thinking of task items (which to some extent is all PIM items with the sometimes exception of contacts) and it is largely how David Allen describes it in his approach.

However, rather than reverting back to the world of mail filters and relying on DnD to file items, we are proposing to "move" items through a life cycle with mutually exclusive "triage status" attributes. The final attribute values are TBD, but the rough proposal is to have 3 states: Processing, Deferred, Done and possible a 4th: Reference to call out items that contain important information that you might actually want to find again later. See Lifecycle workflow below.

---

Workflows

• General workflow principles
• A la carte menu v. Prix fixe Allow the user to explicitly pick the collections behaviors they want as they want them rather than asking them to choose between 2 kinds of collections with preset behaviors that they may or may not understand.
• 004_Comparing_Workflows.gif:
  
• Make sure that users can function entirely within the framework of explicit DnD manipulations. If they want to stay in the current mental model of DnD to move items around, they should be allowed to do so.
• Provide a shallow slope for users to ramp up to intermediate and advanced user functionality: Levels of Use
  1. DnD
  2. Populate with a rule
  3. Model "Lifecycle" workflow with mutually exclusive attributes (ie. Task statii: Todo, Blocked, Done)
  • 3_levels_of_collections.gif:
    

• Another way to slice Levels of Use: Templates
• Users will probably start out with whatever OOTB^? collections Chandler provides. Populating these collections primarily through explicit DnD^? manipulations. We want to provide users with more advances affordances for applying rules to collections, however we want to make sure we provide a shallow learning curve where users can first experiment with easy templated collections before moving onto more complex rule-governed collections that they can tweak. Finally for the most advanced users, there will be affordances for building collections from scratch as well as an integrated UI for filtering on top of a collections base-rule.
• levels_of_use.gif:
  

* Freedom of order We want to allow users to create collections by either

1. CREATE and NAME collection then POPULATE collection or
2. POPULATE collection then NAME and SAVE collection

• 01_wrkflow_bidirectional.gif:
  
• 02_wrkflow_naming_a_rule.gif:
  

Creating templated collections. Users can choose from a selection of templated collections that have preset rules.

• Mailing list filter (Kind=Mail, Mailing list=Auto-generated CollectionName)
• mail_filter_template.gif:
  
• Calendar (Kind=Calendar, Sphere of Life=Home, Work)
• Project (Project=CollectionName)
• Sphere of Life (Sphere of Life=Home, Work, Friends, CollectionName)
• Contact group (Kind=Contact)

• Tweaking templated collections

Modelling an item's lifecycle

• Use cases

1. Inbox as universal archive of all incoming mail
2. Inbox as a selective archive of important mail
3. Inbox as a Dashboard view where only new and processing items live

Levels of use for each use case
The table below describes workflows for each of the use cases above. Each use case is provided with 3 levels of workflows that grow increasingly sophisticated. The workflows are depicted in the storyboards below.

#	Use case	Lvl1: DnD?	Lvl2: Applying rules to populate collections	Lvl3: Modeling Status Lifecycles with Mutually Exclusive attribute values
1	Inbox as universal archive	Hand copy pointers of items to folders	To reply to (PUT IN Mail, Todos) Mailling lists (PUT IN design@) Projects (PUT IN Foo)
2	Inbox as New and Important Email	Hand move items to folders	To reply to (PUT IN Mail, Todos) Mailling lists (PUT IN design@) Projects (PUT IN Foo) Inbox (TAKE OUT Todos, Mailing lists)	Mailing lists (MARK AS Archive) Projects (MARK AS (Archive) Inbox (TAKE OUT Archives)
3	Inbox as Dashboard	Hand move items to folders	To reply to (PUT IN Mail, Todos) Mailling lists (PUT IN design@) Projects (PUT IN Foo) Inbox (TAKE OUT Todos, Mailing lists)	Mailing lists (MARK AS Archive) Projects (MARK AS Archive) Inbox (TAKE OUT Archives)

Caveat The Level 3 workflow: Modeling Lifecycles with mutually exclusive attribute values will be the central workflow of the Dashboard view. The workflow described here is an unecessarily awkward description of how the user might retrofit the Inbox to serve as a surrogate Dashboard view. The Dashboard view will have built in affordances and pre-programmed rules to make it easy for the user to move items through Status lifecycles.

• Workflow_use_cases.gif:
  
• 022_Item_Life_Cycle.gif:
  
• 03_wrkflow_level1_dnd.gif:
  
• 04_wrkflow_level2_setting_r.gif:
  
• 05_wrkflow_level3_me_attrib.gif:
  

Rule builder UI

• rule_builder_close-up.gif:
  
• rule-building-matrix.gif:
  
• complex_rule_based_collecti.gif:
  
• 05_Query_Builder.gif:
  

• Filtering collections
• 04_Collections_Tuning.gif:
  

• Assigning attributes to collections
• 06_Set_Attributes.gif:
  

• Managing attributes
• 08_Attributes_Mgr.gif:
  

• Bonus Setting up positive feedback loop collections
• 09_Collections_Feedback_Loop.gif:
  

---

Structure: Collections and Views

• Proposed assertions (subject to review)
• Collections must always appear in the context of a view
• Each collection can be viewed in a number of different view types depending on what kind of collection it is and what kinds of items it has (ie. calendar)
• Some view settings are specific to a collection and some span all collections or all collections within a collection type

• Use cases for Sharing Collections and Views
• Show and tell, I want to send you a collection or item and walk you through it. In this use case, users will want the Share to look as identical as possible for both Sharer and Sharee.
  • Mitch and Esther want to review the OSAF events calendar together.
• Here's data for you to be aware of and play around with. In this situation, users don't really care to have the same view settings and in fact each member of the Share would prefer to have their own view settings.
  • A travelling salesman sends his travel calendar to his personal assistant so she can book his flights.
• How do we reconcile these two use cases? Proposal: We allow a one-time sharing of view settings and an advanced menu option to re-Sync view settings.

Quick brainstorm about elements of a view that might persist between sessions

View setting	Applies to	Does it override Sharee's view settings once?
Generic view settings
View type selected	same for all calendars, per collection for everything else	Y
Window size	all collections	N
Pane size	all collections	N
What columns are up	per collection type*	Y
Column order	per collection type*	Y
Column sort	per collection	Y
Sort order	per collection with a default for all collections	Y
Column width	all collections with the selected attribute	N
Item selected	per collection	N/A
Selection history	per collection	N
Scrollbar position	per collection	Y
What ad-hoc collections are open	per collection	Y
Font face	all collections	M Sharee has to agree?
Font size	all collections	M Sharee has to agree?
Font color	all collections	M Sharee has to agree?
Hilight color	all collections	M Sharee has to agree?
Timestamp format	all collections	N
Calendar summary view settings
How many weeks	all calendars*	Not if it applies to all calendars**
Absolute or relative time	all calendars*	Not if it applies to all calendars**
Time range	all calendars*	Not if it applies to all calendars**
Timezone	all calendars*	Not if it applies to all calendars**
What you see in the headlines	all calendars*	Not if it applies to all calendars**
Calendar colors	all calendars	Not if it applies to all calendars**

* When do users need the option to customize a view setting for a particular collection? Is that a Canoga feature? **We don't want Sharer's view settings to override the Sharee's view settings if it's a view setting that will affect all collections or all collections of a particular type.

• Collection groupings Where you might want to have the same view settings for all member collections.
• All incoming mail collections
• All outgoing mail collections
• All mixed views
• All of "My calendars"
• Calendars by "owner"
• All project collections
• All sphere of life collections

• collections_own_views.gif:
  

---

Workflows: Collections and Views

---

Feature List

---

Open Issues

• Can we assign attributes to collections? (ie. "Project Foo" is a "Home" project)
• MimiYin, 31 Mar 2004: I think so

• Does "Project Foo" appear as a link to a collection in "Home" or as loose items?
• MimiYin, 31 Mar 2004: Putting "Project Foo" into "Home" as discrete item (link to a collection) has the benefit of providing the user with context. The user remembers labeling "Project Foo" as a member of "Home", not all of "Project Foo's" member items. However, it opens up an entirely different can of worms with respect to hierarchy within a view which we've decided not to address in Canoga. Users can still "filter" by "Project Foo" in the "query by example" filter UI, so in some sense, "Project Foo" still exists as a discrete item in "Home".

• Does DnD mean move or copy a pointer?

• What are the total number of views Canoga should support practically and usably?

• Do we need a separate "View Manager" if we allow for hundreds of views?

• What is the relationship if any between the Home Calendar and the Home Group?

---

Comments Welcome

• MimiYin, 07 Jul, 2004 Why assigning categories to items is not enough for organizing information. Users end up assigning a lot of the same categories to all items. You never know when to stop, since most things are vaguely related to many topics. Users end up with inconsistent labeling where some items are over-labeled and others are under-labeled. Labeling also needs to be prioritized. (ie. This item is primarily about Apes and secondarily about the Congo.) Users often get frustrated or paralyzed because they know they're labeling inconsistently, which makes them want to just forget about it altogether. Having a purely flat, a-hierarchical topic labeling organizational structure also doesn't work because it forces users to do a lot of redundant labeling. (ie. Anything that get's labeled "Sharing Collections" should automatically be labeled "Sharing" as well.) Organizational affordances must be able to reflect hierarchical relationships.

• MimiYin, 15 Jul, 2004 Maybe we don't need to have different types of collections I think for now in .4 and .5 we should certainly try to get away with just having collections and do away with finer distinctions between types of collections (ie. Mailbox, Calendar, Projects). With the Kind filter in the Sidebar, there is already a way to view Collections "as if" there were kind-based collections. Turning on a Kind filter momentarily turns Chandler from a General PIM to a traditiona silo-based application: Email, Tasks, Calendar, Documents. I also have a feeling that Project is just different from Collections and should really be treated more as a descriptive attribute (ie Date, Color) than as a collection.

• MimiYin, 20 Jul, 2004 Special collections Kind filter collections (ie. All communications, All calendar items) and OOTB collections (ie. Dashboard, In, Out, Archive, Junk, Trash, Shares) are all special collections that users cannot delete, tweak with rules and do not show up in the See also section of member items.