Python Logging and Observability

The essentials

Quick reference

One focused task per row. Jump to the related section for complete, working examples.

Use	Syntax	Examples
Create a module logger	`logger = logging.getLogger(__name__)`	View examples
Set a logger threshold	`logger.setLevel(logging.INFO)`	View examples
Defer message formatting	`logger.info('processed %d records', count)`	View examples
Guard expensive diagnostics	`if logger.isEnabledFor(logging.DEBUG): logger.debug('state=%r', build_state())`	View examples
Configure a small application	`logging.basicConfig(level=logging.INFO, format='%(levelname)s:%(name)s:%(message)s')`	View examples
Apply dictionary configuration	`logging.config.dictConfig({'version': 1, 'handlers': handlers, 'root': root})`	View examples
Preserve existing loggers	`config = {'version': 1, 'disable_existing_loggers': False, 'root': root}`	View examples
Add a stream destination	`handler = logging.StreamHandler(sys.stdout)`	View examples
Set a destination threshold	`handler.setLevel(logging.ERROR)`	View examples
Stop duplicate propagation	`logger.propagate = False`	View examples
Attach event context	`logger.info('checkout accepted', extra={'order_id': order_id})`	View examples
Reuse stable context	`request_log = logging.LoggerAdapter(logger, {'request_id': request_id})`	View examples
Record the active exception	`except OSError: logger.exception('configuration load failed')`	View examples
Include exception information	`logger.error('operation failed', exc_info=True)`	View examples
Attribute a helper's caller	`logger.warning('deprecated call', stacklevel=2)`	View examples
Filter one destination	`handler.addFilter(lambda record: record.name != 'app.health')`	View examples
Emit structured records	`handler.setFormatter(JsonFormatter())`	View examples

Useful logs are queryable records of important events, not a transcript of every line executed. Give modules hierarchical loggers, configure destinations at the application boundary, attach stable operational context, preserve exception information, and avoid secrets or unbounded payloads. Logs complement metrics and traces; they should help explain what happened without becoming a second database.

Step by step

Detailed examples

Name loggers by module and let records propagate

getLogger returns the same logger for the same dotted name, and logging.getLogger(__name__) naturally mirrors package structure. A record first passes the originating logger's effective level and then each handler's level. Child records normally propagate to ancestor handlers; centralizing handlers near the root avoids duplicated lines. Use lazy percent-style arguments rather than prebuilt f-strings when formatting or argument construction is avoidable.

Combine logger and handler thresholds

import io
import logging

stream = io.StringIO()
handler = logging.StreamHandler(stream)
handler.setLevel(logging.INFO)
handler.setFormatter(logging.Formatter('%(levelname)s:%(name)s:%(message)s'))
logger = logging.getLogger('demo.service')
logger.handlers.clear()
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)
logger.propagate = False
logger.debug('internal detail')
logger.info('processed %d records', 3)
print(stream.getvalue().strip())

Output

INFO:demo.service:processed 3 records

Back to quick reference ↑

Configure logging once at the application boundary

Libraries should create named loggers but leave destinations and global levels to the application. basicConfig suits small programs; dictConfig expresses larger formatter, filter, handler, and logger graphs. Its disable_existing_loggers option defaults to True, so set it to False unless disabling every unlisted non-root logger is intentional. Treat remotely supplied logging configuration as executable configuration, not untrusted data.

Build a deterministic dictConfig graph

import logging
import logging.config
import sys

logging.config.dictConfig({
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {'plain': {'format': '%(name)s|%(levelname)s|%(message)s'}},
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'formatter': 'plain',
            'level': 'INFO',
            'stream': 'ext://sys.stdout',
        }
    },
    'root': {'level': 'INFO', 'handlers': ['console']},
})
logging.getLogger('app.worker').info('ready')

Output

app.worker|INFO|ready

Back to quick reference ↑

Route records with handler levels and explicit propagation

Handlers decide where accepted records go: streams, rotating files, queues, email, sockets, or system facilities. Logger and handler thresholds both apply, so a DEBUG logger can feed an INFO console and an ERROR alert stream. Attaching handlers to both a child and its ancestor duplicates output when propagation remains enabled. Prefer QueueHandler and QueueListener when slow handler work must stay off latency-sensitive threads.

Send errors to a dedicated destination

import io
import logging

all_events = io.StringIO()
errors = io.StringIO()
console = logging.StreamHandler(all_events)
console.setLevel(logging.WARNING)
error_handler = logging.StreamHandler(errors)
error_handler.setLevel(logging.ERROR)
formatter = logging.Formatter('%(levelname)s:%(message)s')
console.setFormatter(formatter)
error_handler.setFormatter(formatter)
logger = logging.getLogger('routing.demo')
logger.handlers.clear()
logger.addHandler(console)
logger.addHandler(error_handler)
logger.setLevel(logging.DEBUG)
logger.propagate = False
logger.warning('slow')
logger.error('failed')
print('all=' + all_events.getvalue().strip().replace('\n', ','))
print('errors=' + errors.getvalue().strip())

Output

all=WARNING:slow,ERROR:failed
errors=ERROR:failed

Back to quick reference ↑

Attach bounded context to the record, not the prose

The extra argument adds non-reserved fields to a LogRecord, and LoggerAdapter makes stable context reusable across calls. Prefer identifiers such as request_id, tenant_id, operation, and outcome over embedding them in free-form messages. Supply formatter defaults or guarantee every record has required fields. Never log passwords, access tokens, session cookies, full payment data, or arbitrary request bodies; redaction after collection is too late.

Reuse request context with LoggerAdapter

import io
import logging

stream = io.StringIO()
handler = logging.StreamHandler(stream)
handler.setFormatter(logging.Formatter('request=%(request_id)s %(levelname)s %(message)s'))
logger = logging.getLogger('context.demo')
logger.handlers.clear()
logger.addHandler(handler)
logger.setLevel(logging.INFO)
logger.propagate = False
request_log = logging.LoggerAdapter(logger, {'request_id': 'req-7'})
request_log.info('loaded %d items', 2)
print(stream.getvalue().strip())

Output

request=req-7 INFO loaded 2 items

Back to quick reference ↑

Preserve exception identity at the handling boundary

logger.exception is shorthand for an ERROR event with exc_info=True and should be called while handling an active exception. It preserves type, message, and traceback for diagnosis; logging only str(error) discards crucial context. Avoid logging and re-raising the same exception at every layer because each boundary creates another duplicate event. Wrapper helpers can pass stacklevel so source attribution points to their caller.

Format exception metadata deterministically

import io
import logging

class ExceptionFormatter(logging.Formatter):
    def format(self, record):
        kind = record.exc_info[0].__name__ if record.exc_info else '-'
        return f'{record.levelname}:{record.getMessage()}:{kind}'

stream = io.StringIO()
handler = logging.StreamHandler(stream)
handler.setFormatter(ExceptionFormatter())
logger = logging.getLogger('exception.demo')
logger.handlers.clear()
logger.addHandler(handler)
logger.setLevel(logging.ERROR)
logger.propagate = False
try:
    1 / 0
except ZeroDivisionError:
    logger.exception('calculation failed')
print(stream.getvalue().strip())

Output

ERROR:calculation failed:ZeroDivisionError

Back to quick reference ↑

Emit stable structured events and filter close to the destination

A custom Formatter can serialize selected LogRecord fields for ingestion, but define a schema instead of dumping record.__dict__, which contains unstable and potentially sensitive values. Use record.getMessage() to apply deferred arguments. Handler filters are evaluated immediately before that destination emits a record, making them suitable for routing or suppressing noisy health checks; metrics are usually better for high-volume counters.

Serialize selected fields and suppress health noise

import io
import json
import logging

class JsonFormatter(logging.Formatter):
    def format(self, record):
        return json.dumps({
            'level': record.levelname,
            'message': record.getMessage(),
            'service': record.service,
        }, sort_keys=True)

stream = io.StringIO()
handler = logging.StreamHandler(stream)
handler.setFormatter(JsonFormatter())
handler.addFilter(lambda record: record.name != 'app.health')
root = logging.getLogger('app')
root.handlers.clear()
root.addHandler(handler)
root.setLevel(logging.INFO)
root.propagate = False
logging.getLogger('app.health').info('ok', extra={'service': 'api'})
logging.getLogger('app.orders').info('accepted %s', 'A-7', extra={'service': 'api'})
print(stream.getvalue().strip())

Output

{"level": "INFO", "message": "accepted A-7", "service": "api"}

Back to quick reference ↑

Local code tester

Emit contextual application events

Adjust levels, context, and messages while observing the stable record format.

Runs in your browser

Python

import logging
import sys

handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logging.Formatter('%(levelname)s %(name)s request=%(request_id)s %(message)s'))
logger = logging.getLogger('demo.worker')
logger.handlers.clear()
logger.addHandler(handler)
logger.setLevel(logging.INFO)
logger.propagate = False

request_log = logging.LoggerAdapter(logger, {'request_id': 'req-42'})
request_log.info('processed %d items', 3)
request_log.warning('retry budget is low')

Output

Press Run to load Python locally.

Sources and further reading

References

Authoritative documentation used to verify and expand this cheat sheet.

Help us improve

Found a typo or missing example?

Tell us what would make this cheat sheet clearer, more complete, or more useful.

Share feedback

Quick reference

Detailed examples

Name loggers by module and let records propagate

Configure logging once at the application boundary

Route records with handler levels and explicit propagation

Attach bounded context to the record, not the prose

Preserve exception identity at the handling boundary

Emit stable structured events and filter close to the destination

Emit contextual application events

More Python cheat sheets

Python importlib, Metadata, Resources, and Plugins Cheat Sheet

Python Modules and Packages Cheat Sheet

Python Packaging, Virtual Environments, and pyproject.toml Cheat Sheet

References

Found a typo or missing example?