Framing the issue

In pursuit of knowledge...

The Information Age. What does that mean?

Computers didn't invent information. So why is it now that we have computers, information is now Information with a capital I?

Computers didn't invent information, but computers did virtualize information, making it more mutable, transportable and omni-present. Capital-I Information transcend the laws of physics. It also made it possible to capture and record our lives (at least the digital traces of it).

The fallout of that is relatively straightforward.

• Information can be aggregated and reaggregated in infinite combinations and recombinations.
• Heterogeneous types of informations can co-exist happily in the same place. (Try filing a busted carburetor into your "To Fix" folder in your file cabinet?)

• • Email re: Potluck dinner next Thursday.
  • Blog posting about the quality of schools in Westchester county.
  • Washington Post article about heart disease in men over 55.
  • Business contact information for your real estate agent.
  • Biographical data about Richard Strauss.
  • Reminder to put the garbage out on Monday mornings.
  • Itinerary for trip to Copenhagen.
  • Child's report card.
  • Book review of new Bob Dylan poetry reader.
  • Price comparison for snazzy new PDA phones.
  • Directions to movie theater and movie tickets.
  • Rental listings and maps.

• This in turns provides us with a huge opportunity to provide synthesized, big picture views of data that were impossible when information was confined to the laws of physics.

This makes sense, life is heterogeneous...and capital-I Information is only useful (usable) if you can effectively aggregate, process, manage and organize heterogeneous information in one place.

But capital-I Information is also unruly, unpredictable and high volume. And the signal to noise ration can sometimes be distressingly low.

Chandler's goal is to provide an information management that takes advantage of all of the positive characteristics of capital-I Information while keeping at bay it's negative side effects.

The question we must answer then is: How to grok such unruly information?

---

A different way of posing the question is: What is the paradigm shift that is needed to deal with this new breed of capital I-Information?

A member of our community Daniel Vareika posted an article from Business Week called The Real Reasons You're Working So Hard which addressed many themes that resonate deeply with Chandler's design philosophy. Information overload. Inefficient information management tools. It seems we've reached that part of the curve where our wonderful new toy: Information Technology is starting to lose some of it's lustre and starting to actively impede productivity.

Things have come to a head so to speak where we either have to take a leap of faith and shift to a radically different paradigm, one that is better suited to the amorphous, easy-come, easy-go, messy nature of information in the digital age. Or we chug some mountain dew, keep on with our carpal tunnel finger exercises and keep on truckin.

Below is my response to Daniel's article post:

In a paper-based world where you received 1 or 2 or 3 letters a week via Pony Express, the rate at which information arrived was so slow that you could take the time to parse each item individually. And the effort was usually worth it. The scarcity of communication resources was such that people were also incredibly efficient in their communications. If you could only send one letter a month, you made sure it was a letter worth reading.

Today, we are clearly at the opposite extreme. Communications is cheap, so like with most things that are cheap, we're careless with them. SPAM is a big problem, but the lack of an incentive for people to be efficient with their communications has a much more significant negative impact. (It's easy to identify and delete SPAM, but how do you deal with sloppy information that is actually relevant to you?) People send one off emails rather than waiting to compile a list of questions. People address emails to team lists that may have dozens of recipients on them (I'm not talking about discussion lists like this one ;o) rather than carefully calling out the few people that really need to know in the To: field and maybe CC:ing the rest of the list.

Then of course, the information overload feeds on itself. Person A is careless is about the emails they send. Person B gets too many of Person A's emails and can't keep up with all of them. So then Person A has to send more duplicate emails to nag Person B. The net effect of this multiplied over the dozens of people we deal with every day results in a broken Inbox, with or without Spam.

Which leads us to the paradigm shift. Why are we still parsing and processing emails one-by-one, stuck down at the runway level, navigating our information with no perspective and no high level understanding of what's going on?

We need to elevate ourselves a little bit and survey what we have at the 10 or 20,000 foot level. Deal with things with broad brush strokes and then only drill in on the parts we've identified as important.

In part, Chandler deals with this with our notion of Triage (as in, don't meticulously file everything, triage most of it and then only "organize" the important stuff).

We also have the idea of organizing your life into high-level Spheres: Home, Work, School, Fun etc...with Areas of Responsibility or "Projects" in each Sphere.

These are attempts to provide eagle-eye perspectives on your data.

But ultimately, we will need to address this issue of how people can "make high-level" sense of their data without having to open and scan every email one at a time, and this is largely a data visualization issue.**

Scenarios

1. Can I get a high level view of my Inbox where I can see which emails are To: me, which emails are CC: me and which emails are to some list I'm on?
2. Can I get a high level view of my Inbox where I can see the information described in scenario#1 overlayed with groupings based on WHO the email is from?
3. Can I then overlay what project the email is about?

Now, I can start to make intelligent decisions about the order in which I should deal with my Inbox. I'll start with the emails (To: me)+(From:My Manager)+(About:Project due Tomorrow)

And in one fell swoop, I can automatically Archive into the "I'll read this someday maybe" pile, all the emails that are (To: Some list I'm on) or (About:Office Announcements), etc...

Just as we've realized that: Now that we're not literally filing paper into manila folders into steel file cabinets...Therefore, we don't have to restrict items to "live" in one and only one location...

...We also need to realize that: Now that we're not receiving 1 letter a month anymore...We can no longer afford to maintain our incredibly expensive workflow model of parsing information one individual item at a time.

Desperate times call for desperate measures.

Now, let's ask the question again: How do we attain this 10,000 foot, eagle-eye, contextualized view of our data? How do we make sense of all this mess?

Suggestion by Example by SethJohnson:

I would recommend having a good way to understand heterogeneous data in an easy, abstract way, then build solutions out of that. In other places, I've spoken of mindmapping, and particularly of mindmapping with respect to the notion of relations. I extend the traditional notion of relations among flat table entities, to a broader set of abstractions that can be used to represent all relations while thereby providing the basic data management functionalities needed. I refer to this extended notion of a relation as a "context" -- and I explain elsewhere how mindmapping fits into this generic concept that actually fits all applications. Below, I have some sample screens from an interface that uses this set of abstractions. It's not a mindmapping interface, but that would just be a different front end on the same basic data structure. These should give a clearer idea of the genericity of the concepts.

You only need a few abstractions to handle all possible data structures and put them all in the one data structure.

Here's an example.

• Sample "Context" Interface:
  

We see an outlining interface for working on Review Instruments, which are sets of criteria for auditors to review health care quality. Review instruments have multiple Review Criteria, which are outlined in the interface shown above.

This is an example of a context. Its Use Type is Review Instrument; its Link Type is Review Criteria; and its particular instance is Recredentialing Medical Record Review. All of the functionality of manipulating, seeking, editing, distributing, etc. the records -- called Links in this nomenclature -- is automatically available, just by declaring that the context is Review Instruments with Review Criteria.

When you doubleclick on one of the Links, you get:

• Attribute Editing Screen:
  

Here you can see and edit all the attributes and their values relevant for one Review Criterion in a Review Instrument. You can see that the attributes are outlined.

Now, on the main screen again, you see a Search button that looks like an arrow pointing at a line of dots. That produces the following menu:

• Funky Seek Modes:
  

This menu doesn't have an option for seeking by conditional/SQL queries yet, though that's perfectly doable. However, it shows how to change contexts easily. First, you can select from the list of all particular uses of this context (particular review instruments with review criteria):

• Select a Particular Use of the Context:
  

. . . or the subset of review instruments that you've added review criteria to:

• Select a Particular Use You've Worked on:
  

Clicking on those would put you back on the main screen for the particular review instrument. In relational database terms, that puts you in another record of the parent entity. (Note, the physical data structure for this doesn't entail storing entities in their own flat tables -- it's a series of fact tables knitting nodes together in terms of these abstract notions of use types, link types, uses, links, and their respective attributes and categories.)

Now, here's the cool thing. You can change applications instantly, just by picking another use type:

• Change the Use Type:
  

Notice that the use types themselves are outlined for easy organizing and locating. Let's pick the "Position Paper" use type and then pick a link type to complete the definition of our context:

• Change the Link Type:
  

(The seek menu has changed dynamically to show that now you may select Position Papers. It still shows the original Link Type of Review Criteria, and you see a list of other link types you can choose.)

Let's pick the "Paragraph" link type, and since now we have specified the generic context (the use type and link type), we are ready to select one instance of that kind of context:

• Select a Particular Use of the New Context:
  

Now we can pick a particular Position Paper that has Paragraphs in it. Doubleclick the Microsoft DRM OS position paper listed, and you get back to the main context screen, only it's a new application:

• A New Application:
  

This should have more paragraphs in it ("links"), but you get the idea. Now this context has its own attributes that are relevant to it.

What does this have to do with the visualizing aspect of managing heterogeneous information? I present it here in order to give a sense of how generic terms can help simplify understanding while helping see a commonality in concepts that works across possible interfaces. Here, it's pretty simple to see the outline screen could just as well be a mindmap. I believe the interface in these screens is pretty simple and elegant, but I say that solely to illustrate how the context abstraction notions I'm talking about help by providing generic concepts that simplify the interface in themselves.

Design principles such as how much you can hold in your brain and comprehend visually are very important and I applaud your principles. I do suggest, however, that deciding on categories or functions or concepts that are beneficial for design purposes doesn't need to mean that certain interfaces that can represent complexity should be disregarded if they don't appear to be consistent with some set of functions or concepts that are being considered as simplifying techniques for design purposes. The notions in the "context" abstraction provide an avenue for helping solve many problems due to their genericity.

-- SethJohnson - 20 Jan 2006