Warning A lot of the stuff in here is going to be pretty basic, but I figured I'd start at the beginning.

Stating the problem

A continuing source of controversy and contention has been the orthogonal design of the sidebar which imparts a dual-partical-wave feeling on the things or collections in the sidebar. However, I'm wondering if the uneasiness the sidebar provokes in us is actually a result of the complexity that it requires at the implementation level. In other words, the smoke and mirrors required to get the results we want in the UI make the whole thing feel overly complicated and risky.

A second, and by no means mutually exclusive reason why the sidebar feels like it's hard to understand is that perhaps we're looking at it from a traditional lens where things on the computer are static just like things in the real world. It's possible that seen from a different perspective, a more dynamic, free-flowing perspective, the sidebar might start to simplify itself. (In the same vein of how planetary orbits simplified once they were elliptical instead of spherical? OR How really long periods of time and huge expanses of space simplified once time and space became time-space and we switched from Newtonian physics to Einsteinian relativity?)

So what is this new simplified perspective?

The world as we know it in our computers. Things are generally static. Analogies are generally made wrt real, physical world. Files, folders, hierarchical trees. Things are physical things and they appear in one place. Things are defined by 1-dimension: their location.

The world as it's evolving, mostly on the web. However, the internet and some bold and daring desktop apps (ie. iTunes) are making familiar, a different model of things in our computers, a model that is more native to the virtual environment, that takes advantage of the virtual aspect of computers. What's unique about thing on computers is that they aren't actually things, they aren't actually physical. Computer things aren't bound by the laws of physics. Instead, computer things are just ideas. They can live in multiple places. And these places? aren't really places, they're just ideas too. Attributes really or maybe queries.

This is where the user mental model intersects with our back-end architecture. Unfortunately, it's in a back-asswards way. The UI is trying to convince users that things and locations are just ideas, attributes, queries. However under-the-hood, ideas and queries are actually things and locations (aka collections). Where's the leaky abstraction? Let's save that for later.

The purportedly "simple" sidebar mental model

So at the UI level, where items are just ideas and collections are just attributes and queries, then what we have in the sidebar and appbar are not really collections as destinations or locations, but collections as attributes. They're disguised to look and feel similar to traditional PIM appbars and sidebars, but what users are really doing when they click different combinations of sidebar collections and appbar kinds is building a query. (For more detail, see SidebarSpec2)

The entire UI can be thought of as a table. Across the top in the appbar (aka x-axis columns), users can select what Kind of items they want to see: All kinds, Mail, Events or Tasks. Down the left-hand in the sidebar (aka y-axis rows), users can select what Collection of items they want to see: All items, In, Out, Junk, Trash, User-defined collections. In the summary pane is a blown-up view of the cell that is the intersection of the app-bar column selection and sidebar row selection. ie. Here are your Foo collection Tasks...

• grid_sidebar.gif:
  

A wrinkle in the simplicity is that the appbar and sidebar are not independent of each other. There is a uni-directional dependency, where the appbar selection provokes a response in the sidebar to grey out collections or attributes that do not contain any items in that app. In other words, the sidebar reacts to show the user which collections will produce a nullset. However, no such feedback exists in the other direction. Selecting a collection in the sidebar does not provoke feedback in the appbar as to which appbar kinds will produce a nullset.

The "simple" user mental model requires more complex implementation

As it turns out, modeling everything as queries is a bad idea, it would make everything really slow. I'm going to stop here because I shouldn't claim to understand the issue any more than that. So as I've been told, unless we want to take on the research project of reinventing computers, for now, the ideas and queries in the appbar and sidebar are going to actually be things or collections. These collections happen to have queries attached to them, but in essence, they are tangible, unique things with UUIDs.

This means that under-the-hood there are collections in the sidebar and collections in the appbar and the thing in the summary pane is a collection as well. And all of these collections are different...which makes the sidebar start to look really confusing. Traditionally, a user selects something in the sidebar and what they see in the summary pane are the contents of the thing they selected in the sidebar. In Chandler, what the user will see in the summary pane will be a hidden collection that is neither the collection in the sidebar, nor the collection in the appbar, but an invisible collection that is auto-generated by a rule that points to some sidebar collection and some appbar collection. Sounds confusing to the average user.

• collections_table.gif:
  

But maybe it's okay? It's confusing once you try to understand the implementation. But, at the UI level, users should still be thinking of collections as attributes and appbar kinds as filters. Queries that intersect to generate a result set, which appears in the summary pane.

Where the abstraction leaks

Mismatched user mental models and implementation architectures always produce leaks and in this case, they're springing up all over the place. But we're hoping we can do some patchwork because we think the wins at the UI level outweigh the leaks. Basically, we're going have a go at it and see what happens.

Leak #1: Shared collections become asymmetrical User Jumbo selects the appbar kind: Events and then selects the sidebar collection: Home in an attempt to share his Home calendar with his buddy Tiny. In essence he is sharing the cell of his Chandler table which is the intersection of collection:Home and kind:Events. OR You could say he is sharing the result set of the query, collection:Home and kind:Events. Said group of items is uploaded to a WebDAV / CalDAV server and downloaded to user Tiny's Chandler client.

Desired behavior

1. User Tiny can edit the name of User Jumbo's Home to be Jumbo's Home without changing the name of the collection on Jumbo's machine.
2. User Tiny's mental model is that there is an attribute in his sidebar called Jumbo's Home, which when intersected with the appbar Kind filter: Events, produces Jumbo's Home calendar.
3. Jumbo's Home will most probably contain event items that are also stamped as mail and tasks. As a result, Jumbo's Home items will not only appear when intersected with Tiny's Events filter. They will also appear when Jumbo's Home is intersected with the the appbar kinds: All, Mail and Tasks. In other words, Jumbo has shared a bunch of items (a result set) with Tiny, all of which share the common attribute of collection:Home and kind:Events. That means, when these items arrive in Tiny's repository, collection:Home appears in Tiny's sidebar and Tiny's appbar kind filters are free to intersect with collection:Home to produce various results sets, one of which will be the same for both Tiny and Jumbo (collection:Home and kind:Events) and some of which will be different (collection:Home and kind:All, Mail or Tasks).
4. If Jumbo later decides to share the intersection of Home and Tasks, Tiny's Chandler should be able to figure out that this new set of items, still all share the common attribute of collection:Home and should not create a new collection attribute in the sidebar.

What's actually happening Since collections, kinds and what appears in the summary pane are actually all different collections, not queries and results sets...when Jumbo shares the cell, kind:Events intersected with collection:Home, he's actually sharing a physical thing, a collection that is neither the Home collection in the sidebar, nor the Events collection in the sidebar. It's the invisible collection that Jumbo at an explicit level, doesn't even know exists.

When Tiny receives the Share, what he receives is this invisible collection that Jumbo doesn't know about. However, what manifests itself only as a phantom collection in the summary pane for Jumbo is now prominently displayed as an explicit collection in the sidebar for Tiny.

Even worse, the invisible HomeEvents collection that existed in Jumbo's repository pointed to two unique locations or collections: Jumbo's Home collection and Jumbo's Events collection. In Tiny's repository, Jumbo's HomeEvents collection is not only now a sidebar collection, but proceeds to auto-generate 3 more invisible collections that all point to Jumbo's HomeEvents collection and respectively point to Tiny's Mail, Events and Tasks collections. Which means that Tiny's HomeEvents cell is a completely different collection from Jumbo's HomeEvents cell, though theoretically, they should be the same. It's an infinite cascade where invisible collections get promoted to become visible sidebar collections, which in turn generate more invisible collections.

What started out as a relatively simple: Jumbo shares his HomeEvents cell with Tiny, which in turn creates a corresponding HomeEvents cell in Tiny's repository escalates into a far more complex, increasingly assymetricizing process of exchanging invisible collections.

• jumbo_tiny_tables.gif:
  

• Jumbo_Tiny_UI.gif:
  

A further source of confusion which is going to be challenging, even at the mental model level is that Jumbo thinks he's shared the HomeEvents cell of his Chandler Table. However, Tiny, sees shared items, not only when he visits the HomeEvents cell of his Chandler table. He also sees shared items when he has All or Mail or Tasks selected in the appbar. This is due to the cross-silo nature of the Chandler content model which allows users to create hybrid items that are part-mail, part-task and part event. As a result, what Jumbo is sharing isn't so much a discrete cell with hermetically sealed boundaries, but a bag with a semi-porous membrane that overlaps other similarly semi-porous bags. Essentially, Jumbo's table isn't really a table, it's a Venn diagram.

• venn.gif:
  

So what do we do about it

Writing all of this down has helped me to better understand the differing perspectives on this issue. I still believe that the orthogonal organization of the sidebar is the right way to structure collections and navigation for users.

The hunch we've settled on that orthogonality (as opposed to 1-dimensional trees) is crucial to effective information management (it's the big idea behind meta-data) is hard to user test in the traditional sense since its benefits will only be revealed in long-term use. But it's so core to Chandler's mission statement and such a central pillar of the Chandler data model that we feel that it's imperative to reflect it in the UI even though it poses risks.

And in truth, I'm not convinced that a more traditional linear organization of collections in the sidebar would be less risky...simply because it creates the opposite problem where the back-end is more powerful than the UI lets on and the UI, in order to be safe, is designed in such a way as to misrepresent the way things are architected underneath.

Another way of saying it is that there are really 3 layers to consider:

1. UI
2. Implentation of UI
3. Data model

To be continued.