Logging Proposal

Problem Domain

currently in chandler there is a lack of unified logging architecture, which manifests currently in the variety of approaches taken by developers to gain insight into code running in the chandler framework. through the source base numerous mixes of python print statements, and independent uses of the python logging module [1] exist as evidence of this. clearly this leads to issues when attempting to configure the logging for the chandler framework as a whole, and will ill scale as the chandler application grows. this proposal seeks to implement a solution which will unify use of logging in chandler and allow for central configuration of the logging framework.

Logging Use Cases

some common use cases for logging see [2] and [3] for additional it$

Developer Application Insight / Debugging / Diagnosis

inside complex/large frameworks, its hard to know what code is being triggered by a particular operation, logging can help alleviate this in that following the applications logs can yield insight into the application call stack in response to a particular event.

Crude Timing

although better timing tools exist, logging can often substitute a crude evaluation of timing, esp. in the case of a particular problematic method.

Proposed Solution

in order to effect a solution in chandler for unified logging i propose the following.

an application.log module which will export a functionally equivalent getLogger api as appears in the standard logging module. additionally this module will house the code for the following.
log profiles (debug, standard, production). recognizing that one of the benefits of the standard logging module [1] is its configurability, and that the needs for application logging vary both by the applications lifecycle and a developer's preferences, an abstraction is needed for these various configurations. a logging profile, establishes the configuration for the entire logging infrastructure in chandler. the profile encapsulates various channels or paths in the hierarchy of loggers and their accompanying configuration of formatters, filters/levels, and handlers. as an example of setting up a profile:
```
         repo_debug = LogProfile( "repo_debug",

               ChannelInfo(
                  channel_name = '', # root logger
                  level = 'info',
                  ),

               ChannelInfo(
                  channel_name = 'repository',
                  level = 'debug',
                  )

               ChannelInfo(
                  channel_name = "repository.query",
                  level = "debug",
                  format = "",
                  handler = (StreamHandler(sys.stdout), )
                  )
               )

         repo_debug.enable()
```
log profile manager, a simple api to manage the active log profile.
CHANDLER_LOG_PROFILE environment variable, at least for bootstrapping this proposal's implementation, the default profile enabled for the application at startup.
parcel xml will be extended to allow for profile definition and defining default characterstics of channels.. this is definitely a nesc. refinement of this proposal post its initial implementation and bootstrap integration.

Chandler Impact

Application/Parcel Impact

all existing python logging and usage should defer to the configuration specified above, and use the application.log.getLogger api to retrieve a log object:

          from application.log import getLogger
          log = getLogger('osaf.dav')

          log.debug("msg") 
          operation()

Repository Impact

the repository is intended to be a stand alone technology piece for which a dependency on the application code would be problematic. to facilitate unified configuration, an optional argument will be passed to repository creation which will signal whether or not the repository will configure its own logging or instead use the existing environment configuration. the repository will continue to use the std python logging accessors.

Runtime Concerns

there is some minimal associated overhead with the logging mostly as the standard python logging implementation acquires a thread lock to synchronize log generation and emitting to handlers.

a production profile might install a shim method getLogger which will return a null logger which will ignore logging calls for particular channels or globally if this overhead is deemed signifigant, which reduces the cost to a single function call per logging call.

Additional Considerations

reducing the number of magic environment variables chandler depends on, and switching to use of the OptionParser^? module to allow for application command line settings might be a good alternative for setting the default logging profile.

Open Questions

a definitive logging hierarchy would be nice, but its definitely an open question. some tenative possibilities to open discussion:

       - app
          - gui
          - repo
          - mail
          - sync
          - event

       or

        - repo
           - query
        - parcel
           - mail
           - calendar
        - gui
           - event

a consideration in developing a log hierarchy is how third party parcels, should fit in to be considered well behaving.

while putting together this proposal i came across, a previous logging proposal which advocates use of the python std logging framework, the major difference being this proposal provides for a framework for unifying configuration of logging.

Author

kapil thangavelu <hazmat@objectrealms.net> nick hazmat@irc.osafoundation.org/#chandler

thanks also to katie for offering feedback and suggestions

Source

this proposal was written in structuredtext and converted to html for the wiki using plone.org css the raw source of the proposal is available