Evaluating application frameworks

David Surovell

Chandler GUI Frameworks Engineer, OSA Foundation

18 February 2006

Draft 0.15

---

---

---

---

---

Introduction

This document is an incomplete attempt to describe the decisions regarding the adoption of application frameworks in the course of an application development effort. The efforts within are to quantify the inputs into these decisions, along with discussions of relevant practical tools and techniques. Much of the content is presented in a checklist format.

The overall goal of this document is to articulate a general framework selection criteria, to be used as a living guideline for evaluating Chandler's framework requirements.

It is presumed that you, the reader, has some familiarity with at least one of the popular frameworks (Microsoft's .Net and Foundation Classes, Metrowerks PowerPlant, the Standard C++/C libraries, etc.) and their relationship to typical applications development efforts.

---

Why Use a Framework?

In the early phases of application development, most developers usually consider the adoption of one or more software frameworks in which to implement their application code. The primary goal of this decision is to save valuable development time and resources. The general expectation is that this goal will be realized by relying on the framework implementation to implement acceptable generic behaviors over a wide range of feature domains for which the application has an insignificant amount of specialized requirements. For example, a paintbox application developer doesn't want to spend much effort in writing IP security countermeasures. A firewall applications may not need snazzy drag/drop handlers.

An alternate scenario where developers are likely to consider an application framework is when porting an existing application to a new platform. The goal to be realized by this choice is the same as in the new application scenario.

The toolkits provided by platform vendors tend to be broad in scope and relatively low-level, without significant mid-level support. Part of an application framework's charter is to adapt these toolkits to suit an application codebase so that the application codebase has a relatively high ratio of application-specific to application-total code.

Applications usually require linkages to other additional code components. These components belong to neither the application code base nor the framework. Suitable frameworks must be able to accommodate these components easily and without significant degradation.

Is it possible to write an application without using a framework? Certainly it's possible, but more often than not, it's quite impractical.

Briefly, a framework should meet the following requirements:

• narrow and maintain the development focus to core code (IP architecture and implementation)
  • keep the ratio (high) of core code / total code
  • keep the volume (low) of generic non-core code
• insulate the core code development effort from lower-level details and issues
• provide a vehicle for the integration of other code components (good substrate)
• provide options for future platform deployment options
• provide enough flexibility for incremental changes to the development plan
  • e.g., today's application can become tomorrow's plugin

---

Definitions

Before moving on, a bit of terminology is in order:

* Platforms and Feature Domains

• a platform is simply a collection of hardware running some well-known operating system
  • CPU(s), RAM, disk, screen, keyboard, mouse
• a feature domain is a (narrow) well-defined area of functionality
• a platform typically contains a minimum of 20 feature domains; the well-known desktop platforms contain a great deal more (> 100)

* Development Environments, OS Toolkits and SDKs

• a development environment is a suite (collection) of applications needed to develop software for a specific platform
• a toolkit is usually provided by the OS vendor to provide acceptable atomic access to a platform feature domain
• an SDK is a collection of import and linkage source files, some documentation and source code for one or more demo programs
  • the collection of APIs and their respective SDKs
• SDKs are typically provided on a per-toolkit basis
  • often specific to a development environment

* Frameworks and Components

• components and frameworks are a collection of software to designed to improve access to one or more toolkits
• components are slaves to frameworks
• frameworks are peers of other frameworks
  • using multiple frameworks requires coordination: intricate at best; impractical at worst

---

Making the Tally

Eventually, you'll need to summarize and quantify your review of a framework. A framework evaluation typically involves the following steps:

• clarify the needs of the target application:
  • choose the platforms for the target application
  • choose the preferred development environment(s)
  • identify the required components (all 3 levels)
• search for and select one or more suitable frameworks:
  • it's better to have more than one framework, as cross-comparisons are useful
  • browse the user communities and other web resources for dev't commentary, assessments and resources
• evaluate each framework as follows:
  • licensing requirements
  • integration with the target development environment(s)
  • support for integration for required components
  • flexible support within primary feature domains
  • generic behavioral coverage within secondary feature domains
• test each framework as follows:
  • read the "Getting Started" and "Architecture" documentation (if available)
  • browse the code base
  • build some sample applications
  • integrate custom code into the sample application(s)

---

Cross-platform Considerations

A framework that provides cross-platform support has a substantial number of additional requirements, above and beyond the obvious implementation challenges.

• differences in behaviors due to component implementations
• difficulty of feature domains across platforms where the toolkit APIs have differing architectures and/or sub-features.
• some examples:
  • pthread != Win32 thread
  • Mac toolbar != Win32 toolbar
  • Win32 GDI != Mac QuickDraw != Mac Quartz != GTK+
  • event model differences
• byte ordering, integer size

---

Modeling

A framework's architecture should be well-articulated and reasonably easy to comprehend. The design should be readily evident from a structured examination of the data definition language, either from browsing the raw source files (e.g., the ".h", ".i"), or from a higher-level specialized browsing tool, such as a class browser. Typically, frameworks are accompanied by documentation that describe the both the (high-level) design and (lower-level) behavior.

A framework's architecture should fit/map onto or otherwise accommodate the target application architecture: formally (UML, MVC, etc.) or informally. Framework elements of particular concern are those involving UI interactions: all functionality driven by user input (mouse clicks and drags, keystrokes) should be exposed to the level where they can be invoked without the corresponding user input. Applications that meet this criterion are said to be fully-factored and are typically scriptable, with little if any additional engineering required.

---

Feature Domain Accommodation

A Primary Domain is one that is central to the target application. For example:

• graphics for video games
• secure IP services for net clients
• SQL, MySQL for DB apps
• PHP, CSS for forms and documentation apps
• UI services for "commercial-grade" consumer apps.

A Secondary Domain is one where functionality is required, but a generic implementation will suffice. This ends up being nearly everything else.

---

Feature Domains (A Short List)

What follows is a list of familiar feature domains. It's coarse, yet short. Perhaps some important ones have been omitted.

Feature domains can be classified as belonging to one of the following families:

• low-level OS services
• network
• graphics
  • UI-level: basic widget rendering, themes, etc.
  • document-level: solid 2D primitives, graphics devices, colorspaces, etc.
• text + int'l
• UI interaction: windows, widgets and events
• database
• other
  
  
  
  

Let's start with some common (low-level) necessities:

• memory heap management
• reference counting
• process control
• threading (i/o and other)
• performance
• stdlib:: heap memory, io, file, etc.
  
  
  
  
• asynchronous i/o
• device i/o
• file system
• sockets
• synchronization primitives
• data services: XML, etc.
• int'l services: (esp. for text, Unicode)
• IP services and protocols
• device management: USB, FireWire, etc.
  
  
  
  
• data security services
• access security services (ACLs)
• DRM (sigh)
  
  
  
  

The following feature domains come into play when a screen and user input devices are added to the target platform:

• graphics
  • 2D, minimal (cell, iPod, other embedded/small devices)
  • 2D, full (GDI, GDI+, CoreGraphics, GTK+)
  • 3D (OpenGL, Direct3D)
    
    
    
    
  • geometry: data definitions, simple rendering, etc.
  • graphic devices, windows, clipping
  • color: data definitions, colorspaces, color matching, etc.
  • alpha channel, layered rendering
  • fonts
  • printing
    
    
    
    
• User interaction:
  • windows and related events
  • screen management
  • keyboard: focus, navigation, etc.
  • integration with graphics subsystems
    
    
    
    
• UI widgets (child windows and controls)
  • familiar: menu, scrollbar, combobox, static text, edit text, button, lists, etc.
  • event message wiring
  • containment hierarchies
  • layout support
  • clipboard support
  • focus handling
  • drag/drop
  • themes, skins
  • accessibility
    
    
    
    
• text widgets
  • text display and entry (huge issue)
    • standard attribute handling (at least 20 are important)
    • line endings
    • clipboard support
    • i18n
    • HTML, RTF, CSS
  • advanced services
    • sorting, spelling, transliteration / transcoding, indexing, etc.
    • styled (rich) text editor
    • text to/from speech
    • translation
      
      
      
      
• user input and interaction
  • screen management
  • keyboard / keypad / mouse / joystick / other
    • levels:
      • low: raw device i/o
      • medium: InputMethods, window, widget
      • high: property sheet, dialog, canvas, tool palette, etc.

Here are some important advanced services:

• file system browsers:
  • Open, Save; File, Directory, Volume
  • must handle FS locks and other protections
• sound
• media streams
• data services: SQL, XML
• color matching
• spreadsheet / grid / DataBrowser UI
• accessibility support (services for vision- or hand- impaired users)

Finally, the highest level is the application layer. Here are some of services that you can expect to find:

• high-level native application services
  • application startup and shutdown
  • top-level event dispatcher
  • top-level command handlers
• application model conformance
• debug and console management

---

Assessment of coverage

The largest area of work is here: plowing through framework code. Here are some tasks to keep you busy:

• review the code:
  • do the APIs look clean and well-designed? is the namespace relatively consistent and comprehensible?
  • do the data structures and implementations appear to be mature or incomplete?
    • how much code is "under construction": none? a little? a lot?
    • are there any useful comments?
    • does the code appear to follow well-known conventions and best practices?
  • is there good debugging and logging support?
  • cross-platform criteria
    • determine cross-platform architecture (e.g., class hierarchy)
    • determine data structure portability
• examine these key classes:
  • app, string, thread, file
  • everything that defines and implements access to the primary feature domains of the target application
    • keep track of file names and paths
• evaluate OS toolkit compliance and access:
  • native callback support
    • all required callbacks exposed
    • data access properly and fully implemented

Be especially critical and careful when evaluating areas concerning the application's primary feature domains.

Take notes. Make bookmarks.

---

Development environment accommodation and integration

Frameworks are coupled to one or more development environments out of necessity. This relationship is extremely important for a wide variety of reasons. A framework must be well-integrated with a (integrated) development environment (IDE). The primary features of IDEs are as follows:

• source code editor, project/makefile management
• interactive source-level debugging (ISDB)
  • especially from within a component
• source code revision control system
• code architecture and implementation browser
• code quality analyzer

Source code and documentation navigation

Frameworks usually have a large disk footprint. Source code, sample code, documentation and assorted utilities add up quickly. Within the framework source base are large number of classes and datatypes, message maps and other messy macros, templates, etc. For some developers, emacs will take care of everything; for most others, framework must be searchable and "surfable" via language* and structure-sensitive tools. Cross-linked documentation, class/object browsers are some of the more common features available within development environments. Frameworks should integrate smoothly with these tools.

---

User communities

Frameworks typically have a community of users. This group can be formal and/or informal, public and/or secretive, cooperative and/or ill-behaved. In most cases, the associated user communities are a valuable asset and indispensable resource to individual development efforts. Some questions that can resolved through user communities are as follows:

• is the framework in active development?
• is the framework mature?
• are there active users groups interacting with the development efforts
• can you contribute code?
• can you submit bugs, patches and feature requests?

---

Licensing: open source, proprietary or hybrid?

Adopting a framework always involves licensing considerations. Generally speaking, you'll pay a percentange of [# of customers X # per capita revenue]. This doesn't always mean zero cost for free applications.

Licensing issues cut across all code bits: framework, components and the target application's core code. Some license type don't mix easily, or at all. A simple rubric is to find the code components with the most restrictive licenses. You may need legal assistance.