Logging¶
A quick logging primer¶
Django uses Python’s builtinlogging module to perform system logging.The usage of this module is discussed in detail in Python’s own documentation.However, if you’ve never used Python’s logging framework (or even if you have),here’s a quick primer.
The cast of players¶
A Python logging configuration consists of four parts:
Loggers¶
A logger is the entry point into the logging system. Each logger isa named bucket to which messages can be written for processing.
A logger is configured to have alog level. This log level describesthe severity of the messages that the logger will handle. Pythondefines the following log levels:
DEBUG: Low level system information for debugging purposesINFO: General system informationWARNING: Information describing a minor problem that hasoccurred.ERROR: Information describing a major problem that hasoccurred.CRITICAL: Information describing a critical problem that hasoccurred.
Each message that is written to the logger is aLog Record. Each logrecord also has alog level indicating the severity of that specificmessage. A log record can also contain useful metadata that describesthe event that is being logged. This can include details such as astack trace or an error code.
When a message is given to the logger, the log level of the message iscompared to the log level of the logger. If the log level of themessage meets or exceeds the log level of the logger itself, themessage will undergo further processing. If it doesn’t, the messagewill be ignored.
Once a logger has determined that a message needs to be processed,it is passed to aHandler.
Handlers¶
The handler is the engine that determines what happens to each messagein a logger. It describes a particular logging behavior, such aswriting a message to the screen, to a file, or to a network socket.
Like loggers, handlers also have a log level. If the log level of alog record doesn’t meet or exceed the level of the handler, thehandler will ignore the message.
A logger can have multiple handlers, and each handler can have adifferent log level. In this way, it is possible to provide differentforms of notification depending on the importance of a message. Forexample, you could install one handler that forwardsERROR andCRITICAL messages to a paging service, while a second handlerlogs all messages (includingERROR andCRITICAL messages) to afile for later analysis.
Filters¶
A filter is used to provide additional control over which log recordsare passed from logger to handler.
By default, any log message that meets log level requirements will behandled. However, by installing a filter, you can place additionalcriteria on the logging process. For example, you could install afilter that only allowsERROR messages from a particular source tobe emitted.
Filters can also be used to modify the logging record prior to beingemitted. For example, you could write a filter that downgradesERROR log records toWARNING records if a particular set ofcriteria are met.
Filters can be installed on loggers or on handlers; multiple filterscan be used in a chain to perform multiple filtering actions.
Formatters¶
Ultimately, a log record needs to be rendered as text. Formattersdescribe the exact format of that text. A formatter usually consistsof a Python formatting string containingLogRecord attributes; however,you can also write custom formatters to implement specific formatting behavior.
Using logging¶
Once you have configured your loggers, handlers, filters andformatters, you need to place logging calls into your code. Using thelogging framework is very simple. Here’s an example:
# import the logging libraryimportlogging# Get an instance of a loggerlogger=logging.getLogger(__name__)defmy_view(request,arg1,arg):...ifbad_mojo:# Log an error messagelogger.error('Something went wrong!')
And that’s it! Every time thebad_mojo condition is activated, anerror log record will be written.
Naming loggers¶
The call tologging.getLogger() obtains (creating, ifnecessary) an instance of a logger. The logger instance is identifiedby a name. This name is used to identify the logger for configurationpurposes.
By convention, the logger name is usually__name__, the name ofthe python module that contains the logger. This allows you to filterand handle logging calls on a per-module basis. However, if you havesome other way of organizing your logging messages, you can provideany dot-separated name to identify your logger:
# Get an instance of a specific named loggerlogger=logging.getLogger('project.interesting.stuff')
The dotted paths of logger names define a hierarchy. Theproject.interesting logger is considered to be a parent of theproject.interesting.stuff logger; theproject loggeris a parent of theproject.interesting logger.
Why is the hierarchy important? Well, because loggers can be set topropagate their logging calls to their parents. In this way, you candefine a single set of handlers at the root of a logger tree, andcapture all logging calls in the subtree of loggers. A logging handlerdefined in theproject namespace will catch all logging messagesissued on theproject.interesting andproject.interesting.stuff loggers.
This propagation can be controlled on a per-logger basis. Ifyou don’t want a particular logger to propagate to its parents, youcan turn off this behavior.
Making logging calls¶
The logger instance contains an entry method for each of the defaultlog levels:
logger.debug()logger.info()logger.warning()logger.error()logger.critical()
There are two other logging calls available:
logger.log(): Manually emits a logging message with aspecific log level.logger.exception(): Creates anERRORlevel loggingmessage wrapping the current exception stack frame.
Configuring logging¶
Of course, it isn’t enough to just put logging calls into your code.You also need to configure the loggers, handlers, filters andformatters to ensure that logging output is output in a useful way.
Python’s logging library provides several techniques to configurelogging, ranging from a programmatic interface to configuration files.By default, Django uses thedictConfig format.
In order to configure logging, you useLOGGING to define adictionary of logging settings. These settings describes the loggers,handlers, filters and formatters that you want in your logging setup,and the log levels and other properties that you want those componentsto have.
By default, theLOGGING setting is merged withDjango’sdefault logging configuration using thefollowing scheme.
If thedisable_existing_loggers key in theLOGGING dictConfig isset toTrue (which is the default) then all loggers from the defaultconfiguration will be disabled. Disabled loggers are not the same as removed;the logger will still exist, but will silently discard anything logged to it,not even propagating entries to a parent logger. Thus you should be verycareful using'disable_existing_loggers':True; it’s probably not what youwant. Instead, you can setdisable_existing_loggers toFalse andredefine some or all of the default loggers; or you can setLOGGING_CONFIG toNone andhandle logging config yourself.
Logging is configured as part of the general Djangosetup() function.Therefore, you can be certain that loggers are always ready for use in yourproject code.
Examples¶
The full documentation fordictConfig formatis the best source of information about logging configuration dictionaries.However, to give you a taste of what is possible, here are several examples.
First, here’s a simple configuration which writes all logging from thedjango logger to a local file:
LOGGING={'version':1,'disable_existing_loggers':False,'handlers':{'file':{'level':'DEBUG','class':'logging.FileHandler','filename':'/path/to/django/debug.log',},},'loggers':{'django':{'handlers':['file'],'level':'DEBUG','propagate':True,},},}
If you use this example, be sure to change the'filename' path to alocation that’s writable by the user that’s running the Django application.
Second, here’s an example of how to make the logging system print Django’slogging to the console. It may be useful during local development.
By default, this config only sends messages of levelINFO or higher to theconsole (same as Django’s default logging config, except that the default onlydisplays log records whenDEBUG=True). Django does not log many suchmessages. With this config, however, you can also set the environment variableDJANGO_LOG_LEVEL=DEBUG to see all of Django’s debug logging which is veryverbose as it includes all database queries:
importosLOGGING={'version':1,'disable_existing_loggers':False,'handlers':{'console':{'class':'logging.StreamHandler',},},'loggers':{'django':{'handlers':['console'],'level':os.getenv('DJANGO_LOG_LEVEL','INFO'),},},}
Finally, here’s an example of a fairly complex logging setup:
LOGGING={'version':1,'disable_existing_loggers':False,'formatters':{'verbose':{'format':'%(levelname)s%(asctime)s%(module)s%(process)d%(thread)d%(message)s'},'simple':{'format':'%(levelname)s%(message)s'},},'filters':{'special':{'()':'project.logging.SpecialFilter','foo':'bar',},'require_debug_true':{'()':'django.utils.log.RequireDebugTrue',},},'handlers':{'console':{'level':'INFO','filters':['require_debug_true'],'class':'logging.StreamHandler','formatter':'simple'},'mail_admins':{'level':'ERROR','class':'django.utils.log.AdminEmailHandler','filters':['special']}},'loggers':{'django':{'handlers':['console'],'propagate':True,},'django.request':{'handlers':['mail_admins'],'level':'ERROR','propagate':False,},'myproject.custom':{'handlers':['console','mail_admins'],'level':'INFO','filters':['special']}}}
This logging configuration does the following things:
Identifies the configuration as being in ‘dictConfig version 1’format. At present, this is the only dictConfig format version.
Defines two formatters:
simple, that just outputs the log level name (e.g.,DEBUG) and the log message.The
formatstring is a normal Python formatting stringdescribing the details that are to be output on each loggingline. The full list of detail that can be output can befound inFormatter Objects.verbose, that outputs the log level name, the logmessage, plus the time, process, thread and module thatgenerate the log message.
Defines two filters:
project.logging.SpecialFilter, using the aliasspecial. If thisfilter required additional arguments, they can be provided as additionalkeys in the filter configuration dictionary. In this case, the argumentfoowill be given a value ofbarwhen instantiatingSpecialFilter.django.utils.log.RequireDebugTrue, which passes on records whenDEBUGisTrue.
Defines two handlers:
console, aStreamHandler, which prints anyINFO(or higher) message tosys.stderr. This handler uses thesimpleoutput format.mail_admins, anAdminEmailHandler, which emails anyERROR(or higher) message to the siteADMINS. This handler uses thespecialfilter.
Configures three loggers:
django, which passes all messages to theconsolehandler.django.request, which passes allERRORmessages tothemail_adminshandler. In addition, this logger ismarked tonot propagate messages. This means that logmessages written todjango.requestwill not be handledby thedjangologger.myproject.custom, which passes all messages atINFOor higher that also pass thespecialfilter to twohandlers – theconsole, andmail_admins. Thismeans that allINFOlevel messages (or higher) will beprinted to the console;ERRORandCRITICALmessages will also be output via email.
Custom logging configuration¶
If you don’t want to use Python’s dictConfig format to configure yourlogger, you can specify your own configuration scheme.
TheLOGGING_CONFIG setting defines the callable that willbe used to configure Django’s loggers. By default, it points atPython’slogging.config.dictConfig() function. However, if you want touse a different configuration process, you can use any other callablethat takes a single argument. The contents ofLOGGING willbe provided as the value of that argument when logging is configured.
Disabling logging configuration¶
If you don’t want to configure logging at all (or you want to manuallyconfigure logging using your own approach), you can setLOGGING_CONFIG toNone. This will disable theconfiguration process forDjango’s default logging. Here’s an example that disables Django’slogging configuration and then manually configures logging:
LOGGING_CONFIG=Noneimportlogging.configlogging.config.dictConfig(...)
SettingLOGGING_CONFIG toNone only means that the automaticconfiguration process is disabled, not logging itself. If you disable theconfiguration process, Django will still make logging calls, falling back towhatever default logging behavior is defined.
Django’s logging extensions¶
Django provides a number of utilities to handle the uniquerequirements of logging in Web server environment.
Loggers¶
Django provides several built-in loggers.
django¶
The catch-all logger for messages in thedjango hierarchy. No messages areposted using this name but instead using one of the loggers below.
django.request¶
Log messages related to the handling of requests. 5XX responses areraised asERROR messages; 4XX responses are raised asWARNINGmessages.
Messages to this logger have the following extra context:
status_code: The HTTP response code associated with therequest.request: The request object that generated the loggingmessage.
django.server¶
Log messages related to the handling of requests received by the server invokedby therunserver command. HTTP 5XX responses are logged asERRORmessages, 4XX responses are logged asWARNING messages, and everything elseis logged asINFO.
Messages to this logger have the following extra context:
status_code: The HTTP response code associated with the request.request: The request object that generated the logging message.
django.template¶
Log messages related to the rendering of templates.
- Missing context variables are logged as
DEBUGmessages. - Uncaught exceptions raised during the rendering of an
{%include%}are logged asWARNINGmessages whendebug mode is off (helpful since{%include%}silences the exception andreturns an empty string in that case).
django.db.backends¶
Messages relating to the interaction of code with the database. For example,every application-level SQL statement executed by a request is logged at theDEBUG level to this logger.
Messages to this logger have the following extra context:
duration: The time taken to execute the SQL statement.sql: The SQL statement that was executed.params: The parameters that were used in the SQL call.
For performance reasons, SQL logging is only enabled whensettings.DEBUG is set toTrue, regardless of the logginglevel or handlers that are installed.
This logging does not include framework-level initialization (e.g.SETTIMEZONE) or transaction management queries (e.g.BEGIN,COMMIT, andROLLBACK). Turn on query logging in your database if youwish to view all database queries.
django.security.*¶
The security loggers will receive messages on any occurrence ofSuspiciousOperation and other security-relatederrors. There is a sub-logger for each subtype of security error, including allSuspiciousOperations. The level of the log event depends on where theexception is handled. Most occurrences are logged as a warning, whileanySuspiciousOperation that reaches the WSGI handler will be logged as anerror. For example, when an HTTPHost header is included in a request froma client that does not matchALLOWED_HOSTS, Django will return a 400response, and an error message will be logged to thedjango.security.DisallowedHost logger.
These log events will reach thedjango logger by default, which mails errorevents to admins whenDEBUG=False. Requests resulting in a 400 response dueto aSuspiciousOperation will not be logged to thedjango.requestlogger, but only to thedjango.security logger.
To silence a particular type ofSuspiciousOperation, you can override thatspecific logger following this example:
'handlers':{'null':{'class':'logging.NullHandler',},},'loggers':{'django.security.DisallowedHost':{'handlers':['null'],'propagate':False,},},
Otherdjango.security loggers not based onSuspiciousOperation are:
django.security.csrf: ForCSRF failures.
django.db.backends.schema¶
Logs the SQL queries that are executed during schema changes to the database bythemigrations framework. Note that it won’t log thequeries executed byRunPython.Messages to this logger haveparams andsql in their extra context (butunlikedjango.db.backends, not duration). The values have the same meaningas explained indjango.db.backends.
Handlers¶
Django provides one log handler in addition to those provided by thePython logging module.
- class
AdminEmailHandler(include_html=False,email_backend=None)[source]¶ This handler sends an email to the site
ADMINSfor each logmessage it receives.If the log record contains a
requestattribute, the full detailsof the request will be included in the email. The email subject willinclude the phrase “internal IP” if the client’s IP address is in theINTERNAL_IPSsetting; if not, it will include “EXTERNAL IP”.If the log record contains stack trace information, that stacktrace will be included in the email.
The
include_htmlargument ofAdminEmailHandleris used tocontrol whether the traceback email includes an HTML attachmentcontaining the full content of the debug Web page that would have beenproduced ifDEBUGwereTrue. To set this value in yourconfiguration, include it in the handler definition fordjango.utils.log.AdminEmailHandler, like this:'handlers':{'mail_admins':{'level':'ERROR','class':'django.utils.log.AdminEmailHandler','include_html':True,}},
Note that this HTML version of the email contains a full traceback,with names and values of local variables at each level of the stack, plusthe values of your Django settings. This information is potentially verysensitive, and you may not want to send it over email. Consider usingsomething such asSentry to get the best of both worlds – therich information of full tracebacks plus the security ofnot sending theinformation over email. You may also explicitly designate certainsensitive information to be filtered out of error reports – learn more onFiltering error reports.
By setting the
email_backendargument ofAdminEmailHandler, theemail backend that is being used by thehandler can be overridden, like this:'handlers':{'mail_admins':{'level':'ERROR','class':'django.utils.log.AdminEmailHandler','email_backend':'django.core.mail.backends.filebased.EmailBackend',}},
By default, an instance of the email backend specified in
EMAIL_BACKENDwill be used.send_mail(subject,message,*args,**kwargs)[source]¶Sends emails to admin users. To customize this behavior, you cansubclass the
AdminEmailHandlerclass andoverride this method.
Filters¶
Django provides some log filters in addition to those provided by the Pythonlogging module.
- class
CallbackFilter(callback)[source]¶ This filter accepts a callback function (which should accept a singleargument, the record to be logged), and calls it for each record thatpasses through the filter. Handling of that record will not proceed if thecallback returns False.
For instance, to filter out
UnreadablePostError(raised when a user cancels an upload) from the admin emails, you wouldcreate a filter function:fromdjango.httpimportUnreadablePostErrordefskip_unreadable_post(record):ifrecord.exc_info:exc_type,exc_value=record.exc_info[:2]ifisinstance(exc_value,UnreadablePostError):returnFalsereturnTrue
and then add it to your logging config:
'filters':{'skip_unreadable_posts':{'()':'django.utils.log.CallbackFilter','callback':skip_unreadable_post,}},'handlers':{'mail_admins':{'level':'ERROR','filters':['skip_unreadable_posts'],'class':'django.utils.log.AdminEmailHandler'}},
- class
RequireDebugFalse[source]¶ This filter will only pass on records when settings.DEBUG is False.
This filter is used as follows in the default
LOGGINGconfiguration to ensure that theAdminEmailHandleronly sendserror emails to admins whenDEBUGisFalse:'filters':{'require_debug_false':{'()':'django.utils.log.RequireDebugFalse',}},'handlers':{'mail_admins':{'level':'ERROR','filters':['require_debug_false'],'class':'django.utils.log.AdminEmailHandler'}},
- class
RequireDebugTrue[source]¶ This filter is similar to
RequireDebugFalse, except that records arepassed only whenDEBUGisTrue.
Django’s default logging configuration¶
By default, Django configures the following logging:
WhenDEBUG isTrue:
- The
djangologger sends messages in thedjangohierarchy (exceptdjango.server) at theINFOlevel or higher to the console.
WhenDEBUG isFalse:
- The
djangologger sends messages in thedjangohierarchy (exceptdjango.server) withERRORorCRITICALlevel toAdminEmailHandler.
Independent of the value ofDEBUG:
- Thedjango.server logger sends messages at the
INFOlevelor higher to the console.
See alsoConfiguring logging to learn how you cancomplement or replace this default logging configuration.
