Logging¶
Python programmers will often useprint() in their code as a quick andconvenient debugging tool. Using the logging framework is only a little moreeffort than that, but it’s much more elegant and flexible. As well as beinguseful for debugging, logging can also provide you with more - and betterstructured - information about the state and health of your application.
Overview¶
Django uses and extends Python’s builtinlogging module to performsystem logging. This module is discussed in detail in Python’s owndocumentation; this section provides a quick overview.
The cast of players¶
A Python logging configuration consists of four parts:
Loggers¶
Alogger is the entry point into the logging system. Each logger is a namedbucket to which messages can be written for processing.
A logger is configured to have alog level. This log level describesthe severity of the messages that the logger will handle. Pythondefines the following log levels:
DEBUG: Low level system information for debugging purposesINFO: General system informationWARNING: Information describing a minor problem that hasoccurred.ERROR: Information describing a major problem that hasoccurred.CRITICAL: Information describing a critical problem that hasoccurred.
Each message that is written to the logger is aLog Record. Each logrecord also has alog level indicating the severity of that specificmessage. A log record can also contain useful metadata that describesthe event that is being logged. This can include details such as astack trace or an error code.
When a message is given to the logger, the log level of the message iscompared to the log level of the logger. If the log level of themessage meets or exceeds the log level of the logger itself, themessage will undergo further processing. If it doesn’t, the messagewill be ignored.
Once a logger has determined that a message needs to be processed,it is passed to aHandler.
Handlers¶
Thehandler is the engine that determines what happens to each messagein a logger. It describes a particular logging behavior, such aswriting a message to the screen, to a file, or to a network socket.
Like loggers, handlers also have a log level. If the log level of alog record doesn’t meet or exceed the level of the handler, thehandler will ignore the message.
A logger can have multiple handlers, and each handler can have adifferent log level. In this way, it is possible to provide differentforms of notification depending on the importance of a message. Forexample, you could install one handler that forwardsERROR andCRITICAL messages to a paging service, while a second handlerlogs all messages (includingERROR andCRITICAL messages) to afile for later analysis.
Filters¶
Afilter is used to provide additional control over which log recordsare passed from logger to handler.
By default, any log message that meets log level requirements will behandled. However, by installing a filter, you can place additionalcriteria on the logging process. For example, you could install afilter that only allowsERROR messages from a particular source tobe emitted.
Filters can also be used to modify the logging record prior to beingemitted. For example, you could write a filter that downgradesERROR log records toWARNING records if a particular set ofcriteria are met.
Filters can be installed on loggers or on handlers; multiple filterscan be used in a chain to perform multiple filtering actions.
Formatters¶
Ultimately, a log record needs to be rendered as text.Formattersdescribe the exact format of that text. A formatter usually consistsof a Python formatting string containingLogRecord attributes; however,you can also write custom formatters to implement specific formatting behavior.
Security implications¶
The logging system handles potentially sensitive information. For example, thelog record may contain information about a web request or a stack trace, whilesome of the data you collect in your own loggers may also have securityimplications. You need to be sure you know:
what information is collected
where it will subsequently be stored
how it will be transferred
who might have access to it.
To help control the collection of sensitive information, you can explicitlydesignate certain sensitive information to be filtered out of error reports –read more about how tofilter error reports.
AdminEmailHandler¶
The built-inAdminEmailHandler deserves a mention inthe context of security. If itsinclude_html option is enabled, the emailmessage it sends will contain a full traceback, with names and values of localvariables at each level of the stack, plus the values of your Django settings(in other words, the same level of detail that is exposed in a web page whenDEBUG isTrue).
It’s generally not considered a good idea to send such potentially sensitiveinformation over email. Consider instead using one of the many third-partyservices to which detailed logs can be sent to get the best of multiple worlds– the rich information of full tracebacks, clear management of who is notifiedand has access to the information, and so on.
Configuring logging¶
Python’s logging library provides several techniques to configurelogging, ranging from a programmatic interface to configuration files.By default, Django uses thedictConfig format.
In order to configure logging, you useLOGGING to define adictionary of logging settings. These settings describe the loggers,handlers, filters and formatters that you want in your logging setup,and the log levels and other properties that you want those componentsto have.
By default, theLOGGING setting is merged withDjango’sdefault logging configuration using thefollowing scheme.
If thedisable_existing_loggers key in theLOGGING dictConfig isset toTrue (which is thedictConfig default if the key is missing)then all loggers from the default configuration will be disabled. Disabledloggers are not the same as removed; the logger will still exist, but willsilently discard anything logged to it, not even propagating entries to aparent logger. Thus you should be very careful using'disable_existing_loggers':True; it’s probably not what you want. Instead,you can setdisable_existing_loggers toFalse and redefine some or allof the default loggers; or you can setLOGGING_CONFIG toNoneandhandle logging config yourself.
Logging is configured as part of the general Djangosetup() function.Therefore, you can be certain that loggers are always ready for use in yourproject code.
Examples¶
The full documentation fordictConfig formatis the best source of information about logging configuration dictionaries.However, to give you a taste of what is possible, here are several examples.
To begin, here’s a small configuration that will allow you to output all logmessages to the console:
settings.py¶importosLOGGING={"version":1,"disable_existing_loggers":False,"handlers":{"console":{"class":"logging.StreamHandler",},},"root":{"handlers":["console"],"level":"WARNING",},}
This configures the parentroot logger to send messages with theWARNING level and higher to the console handler. By adjusting the level toINFO orDEBUG you can display more messages. This may be useful duringdevelopment.
Next we can add more fine-grained logging. Here’s an example of how to make thelogging system print more messages from just thedjango namedlogger:
settings.py¶importosLOGGING={"version":1,"disable_existing_loggers":False,"handlers":{"console":{"class":"logging.StreamHandler",},},"root":{"handlers":["console"],"level":"WARNING",},"loggers":{"django":{"handlers":["console"],"level":os.getenv("DJANGO_LOG_LEVEL","INFO"),"propagate":False,},},}
By default, this config sends messages from thedjango logger of levelINFO or higher to the console. This is the same level as Django’s defaultlogging config, except that the default config only displays log records whenDEBUG=True. Django does not log many suchINFO level messages. Withthis config, however, you can also set the environment variableDJANGO_LOG_LEVEL=DEBUG to see all of Django’s debug logging which is veryverbose as it includes all database queries.
You don’t have to log to the console. Here’s a configuration which writes alllogging from thedjango named logger to a local file:
settings.py¶LOGGING={"version":1,"disable_existing_loggers":False,"handlers":{"file":{"level":"DEBUG","class":"logging.FileHandler","filename":"/path/to/django/debug.log",},},"loggers":{"django":{"handlers":["file"],"level":"DEBUG","propagate":True,},},}
If you use this example, be sure to change the'filename' path to alocation that’s writable by the user that’s running the Django application.
Finally, here’s an example of a fairly complex logging setup:
settings.py¶LOGGING={"version":1,"disable_existing_loggers":False,"formatters":{"verbose":{"format":"{levelname}{asctime}{module}{process:d}{thread:d}{message}","style":"{",},"simple":{"format":"{levelname}{message}","style":"{",},},"filters":{"special":{"()":"project.logging.SpecialFilter","foo":"bar",},"require_debug_true":{"()":"django.utils.log.RequireDebugTrue",},},"handlers":{"console":{"level":"INFO","filters":["require_debug_true"],"class":"logging.StreamHandler","formatter":"simple",},"mail_admins":{"level":"ERROR","class":"django.utils.log.AdminEmailHandler","filters":["special"],},},"loggers":{"django":{"handlers":["console"],"propagate":True,},"django.request":{"handlers":["mail_admins"],"level":"ERROR","propagate":False,},"myproject.custom":{"handlers":["console","mail_admins"],"level":"INFO","filters":["special"],},},}
This logging configuration does the following things:
Identifies the configuration as being in ‘dictConfig version 1’format. At present, this is the only dictConfig format version.
Defines two formatters:
simple, that outputs the log level name (e.g.,DEBUG) and the logmessage.The
formatstring is a normal Python formatting stringdescribing the details that are to be output on each loggingline. The full list of detail that can be output can befound inFormatter Objects.verbose, that outputs the log level name, the logmessage, plus the time, process, thread and module thatgenerate the log message.
Defines two filters:
project.logging.SpecialFilter, using the aliasspecial. If thisfilter required additional arguments, they can be provided as additionalkeys in the filter configuration dictionary. In this case, the argumentfoowill be given a value ofbarwhen instantiatingSpecialFilter.django.utils.log.RequireDebugTrue, which passes on records whenDEBUGisTrue.
Defines two handlers:
console, aStreamHandler, which prints anyINFO(or higher) message tosys.stderr. This handler uses thesimpleoutput format.mail_admins, anAdminEmailHandler, whichemails anyERROR(or higher) message to the siteADMINS.This handler uses thespecialfilter.
Configures three loggers:
django, which passes all messages to theconsolehandler.django.request, which passes allERRORmessages tothemail_adminshandler. In addition, this logger ismarked tonot propagate messages. This means that logmessages written todjango.requestwill not be handledby thedjangologger.myproject.custom, which passes all messages atINFOor higher that also pass thespecialfilter to twohandlers – theconsole, andmail_admins. Thismeans that allINFOlevel messages (or higher) will beprinted to the console;ERRORandCRITICALmessages will also be output via email.
Custom logging configuration¶
If you don’t want to use Python’s dictConfig format to configure yourlogger, you can specify your own configuration scheme.
TheLOGGING_CONFIG setting defines the callable that willbe used to configure Django’s loggers. By default, it points atPython’slogging.config.dictConfig() function. However, if you want touse a different configuration process, you can use any other callablethat takes a single argument. The contents ofLOGGING willbe provided as the value of that argument when logging is configured.
Disabling logging configuration¶
If you don’t want to configure logging at all (or you want to manuallyconfigure logging using your own approach), you can setLOGGING_CONFIG toNone. This will disable theconfiguration process forDjango’s default logging.
SettingLOGGING_CONFIG toNone only means that the automaticconfiguration process is disabled, not logging itself. If you disable theconfiguration process, Django will still make logging calls, falling back towhatever default logging behavior is defined.
Here’s an example that disables Django’s logging configuration and thenmanually configures logging:
settings.py¶LOGGING_CONFIG=Noneimportlogging.configlogging.config.dictConfig(...)
Note that the default configuration process only callsLOGGING_CONFIG once settings are fully-loaded. In contrast, manuallyconfiguring the logging in your settings file will load your logging configimmediately. As such, your logging config must appearafter any settings onwhich it depends.
