Logging

https://farm5.staticflickr.com/4246/35254379756_c9fe23f843_k_d.jpg

Thelogging module has been a part of Python’s Standard Library sinceversion 2.3. It is succinctly described inPEP 282. The documentationis notoriously hard to read, except for thebasic logging tutorial.

Logging serves two purposes:

  • Diagnostic logging records events related to the application’soperation. If a user calls in to report an error, for example, the logscan be searched for context.
  • Audit logging records events for business analysis. A user’stransactions can be extracted and combined with other user details forreports or to optimize a business goal.

... or Print?

The only time thatprint is a better option than logging is whenthe goal is to display a help statement for a command line application.Other reasons why logging is better thanprint:

  • Thelog record, which is created with every logging event, containsreadily available diagnostic information such as the file name, full path,function, and line number of the logging event.
  • Events logged in included modules are automatically accessible via the rootlogger to your application’s logging stream, unless you filter them out.
  • Logging can be selectively silenced by using the methodlogging.Logger.setLevel() or disabled by setting the attributelogging.Logger.disabled toTrue.

Logging in a Library

Notes forconfiguring logging for a library are in thelogging tutorial. Because theuser, not the library, shoulddictate what happens when a logging event occurs, one admonition bearsrepeating:

Note

It is strongly advised that you do not add any handlers other thanNullHandler to your library’s loggers.

Best practice when instantiating loggers in a library is to only create themusing the__name__ global variable: thelogging module creates ahierarchy of loggers using dot notation, so using__name__ ensuresno name collisions.

Here is an example of best practice from therequests source – placethis in your__init__.py

importlogginglogging.getLogger(__name__).addHandler(logging.NullHandler())

Logging in an Application

Thetwelve factor app, an authoritative referencefor good practice in application development, contains a section onlogging best practice. It emphaticallyadvocates for treating log events as an event stream, and forsending that event stream to standard output to be handled by theapplication environment.

There are at least three ways to configure a logger:

  • Using an INI-formatted file:
    • Pro: possible to update configuration while running using thefunctionlogging.config.listen() to listen on a socket.
    • Con: less control (e.g. custom subclassed filters or loggers)than possible when configuring a logger in code.
  • Using a dictionary or a JSON-formatted file:
    • Pro: in addition to updating while running, it is possible to loadfrom a file using thejson module, in the standard library sincePython 2.6.
    • Con: less control than when configuring a logger in code.
  • Using code:
    • Pro: complete control over the configuration.
    • Con: modifications require a change to source code.

Example Configuration via an INI File

Let us say the file is namedlogging_config.ini.More details for the file format are in thelogging configurationsection of thelogging tutorial.

[loggers]keys=root[handlers]keys=stream_handler[formatters]keys=formatter[logger_root]level=DEBUGhandlers=stream_handler[handler_stream_handler]class=StreamHandlerlevel=DEBUGformatter=formatterargs=(sys.stderr,)[formatter_formatter]format=%(asctime)s %(name)-12s %(levelname)-8s %(message)s

Then uselogging.config.fileConfig() in the code:

importloggingfromlogging.configimportfileConfigfileConfig('logging_config.ini')logger=logging.getLogger()logger.debug('often makes a very good meal of%s','visiting tourists')

Example Configuration via a Dictionary

As of Python 2.7, you can use a dictionary with configuration details.PEP 391 contains a list of the mandatory and optional elements inthe configuration dictionary.

importloggingfromlogging.configimportdictConfiglogging_config=dict(version=1,formatters={'f':{'format':'%(asctime)s%(name)-12s%(levelname)-8s%(message)s'}},handlers={'h':{'class':'logging.StreamHandler','formatter':'f','level':logging.DEBUG}},root={'handlers':['h'],'level':logging.DEBUG,},)dictConfig(logging_config)logger=logging.getLogger()logger.debug('often makes a very good meal of%s','visiting tourists')

Example Configuration Directly in Code

importlogginglogger=logging.getLogger()handler=logging.StreamHandler()formatter=logging.Formatter('%(asctime)s%(name)-12s%(levelname)-8s%(message)s')handler.setFormatter(formatter)logger.addHandler(handler)logger.setLevel(logging.DEBUG)logger.debug('often makes a very good meal of%s','visiting tourists')