Logging Module - Setup, Files, Formatting

Python's Logging Module

the logging module is part of the standard library
provides tracking for events that occur during the runtime of an application
the simplest way to access a logger is to use the root logger
call the logging methods directly

import logging
logging.basicConfig(level=logging.DEBUG)
logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

DEBUG	detailed information typically of interest only when diagnosing problems
INFO	confirmation that things are working as expected
ARNING	an indication that something unexpected happened, or indicates some problem in the near future the software is still working as expected
ERROR	a more serious problem the software has not been able to perform some functions
CRITICAL	a serious error the program itself may be unable to continue running
EXCEPTION	an exception was raised< signals death of the program.

Setting Up a Logger

root logger not recommended for larger applicationss
can create a new logger by calling the getLogger method and passing in the name of the logger
creates a new logger with the name passed in
common to use __name__ as the name of the logger
will create a logger with the name of the module that the logger is created in

import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

logger.debug('This is a debug message')
logger.info('This is an info message')
logger.warning('This is a warning message')
logger.error('This is an error message')
logger.critical('This is a critical message')

Writing to a File

by default the logging module logs messages to the console
can easily configure the logger to write to a file
do so by using the FileHandler class

import logging

logger = logging.getLogger(__name__)

file_handler = logging.FileHandler('app.log')
logger.addHandler(file_handler)

can also create a handler that will rotate the log file when it reaches a certain size by using the RotatingFileHandler
its use is recommended when the app will run for long periods of time

import logging.handlers

logger = logging.getLogger(__name__)

file_handler = logging.handlers.RotatingFileHandler('app.log', maxBytes=1000, backupCount=3)
logger.addHandler(file_handler)

logger allows defining multiple handlers
can write to the console and to a file at the same time

import logging

logger = logging.getLogger(__name__)

file_handler = logging.FileHandler('app.log')
console_handler = logging.StreamHandler()

logger.addHandler(file_handler)
logger.addHandler(console_handler)

Formatting the Log Messages

can format the log messages to include more information such as

the time the message was logged
the name of the logger
the log level
the message itself

import logging

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter(
        '%(asctime)s - %(name)s - %(levelname)s - %(message)s',
        datefmt='%Y-%m-%d %H:%M:%S'
)

stream_handler = logging.StreamHandler()
stream_handler.setFormatter(formatter)

logger.addHandler(stream_handler)
logger.debug('This is a debug message')

the logging module provides a number of attributes which can be included in the log messages

%(asctime)s	the time the message was logger
%(name)s	name of the logger
%(levelname)s	log level
%(message)s	the messaage
%(filename)s	the filename of the module that the logger was created in
%(lineno)d	the line number of the module that the logger was created in
%(funcName)s	the name of the function that the logger is created in

Filtering Log Messages

can apply various filters to the log messages to only log messages that meet certain criteria

import logging

class ImportantFilter(logging.Filter):
    def filter(self, record):
	    return 'important' in record.getMessage()

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger('my_logger')

logger.addFilter(ImportantFilter())
logger.info('This is a regular message')
logger.info('This is an important message')

filter will only log messages that are of the INFO level or above (WARNING, ERROR) and contain the word 'important' in the message

Just Need a Simple Logger?

a tiny little logger with no frills

import logging
logging.basicConfig(
    level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S',
    filename='apply.log'
    )