current position:Home>Loguru: the ultimate Python log solution

Loguru: the ultimate Python log solution

2022-01-31 02:43:50 Brother K reptile

Pay attention to WeChat public number :K Brother reptile ,QQ Communication group :808574309, Continue to share advanced crawler 、JS/ Android reverse and other technology dry goods !

The importance of a journal

The role of logs is very important , Logs can record user actions 、 Exceptions to the program , It can also provide basis for data analysis , The purpose of log is to record errors during the running process of the program , Convenient for maintenance and debugging , Can quickly locate the wrong place , Reduce maintenance costs . Every programmer should know , Not for logging purposes , The log is not random . To realize the process that can restore the whole program execution only through the log file , To be able to see the implementation of the program transparently , Every thread 、 The purpose of each process . The log is like a plane's black box , It shall be possible to recover the whole site and even the details of the abnormality !

Common logging methods

print()

The most common is to put the output function print() As a way of logging , Print all kinds of prompt information directly , Common in personal exercises , It's usually too lazy to configure logs separately , And the project is too small to require log information , No need to go online , No need for continuous operation , It is not recommended to print log information directly for complete projects , In reality, few people do so .

Self writing template

We can see in many small projects that the author has written a log template , Usually use print() perhaps sys.stdout A little encapsulation can achieve simple log output , there sys.stdout yes Python Standard output stream in ,print() The function is right sys.stdout The advanced packaging of , When we're in Python Call the print object in the print(obj) When , It's actually called sys.stdout.write(obj+'\n'),print() The content is printed to the console , And then I added a newline character \n.

The self writing log template is suitable for small projects , You can write templates according to your preferences , No need for too many complex configurations , Convenient and quick , However, this way of logging is not very standardized , Maybe you think the reading experience is good , However, when others contact your project, they often need to spend some time learning the logic of the log 、 Format 、 Output mode, etc , This method is also not recommended for larger projects .

An example of a simple self writing log template :

Log template log.py:

import sys
import traceback
import datetime


def getnowtime():
    return datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")


def _log(content, level, *args):
    sys.stdout.write("%s - %s - %s\n" % (getnowtime(), level, content))
    for arg in args:
        sys.stdout.write("%s\n" % arg)


def debug(content, *args):
    _log(content, 'DEBUG', *args)


def info(content, *args):
    _log(content, 'INFO', *args)


def warn(content, *args):
    _log(content, 'WARN', *args)


def error(content, *args):
    _log(content, 'ERROR', *args)


def exception(content):
    sys.stdout.write("%s - %s\n" % (getnowtime(), content))
    traceback.print_exc(file=sys.stdout)
 Copy code 

Call the log module :

import log


log.info("This is log info!")
log.warn("This is log warn!")
log.error("This is log error!")
log.debug("This is log debug!")

people_info = {"name": "Bob", "age": 20}

try:
    gender = people_info["gender"]
except Exception as error:
    log.exception(error)
 Copy code 

Log output :

2021-10-19 09:50:58 - INFO - This is log info!
2021-10-19 09:50:58 - WARN - This is log warn!
2021-10-19 09:50:58 - ERROR - This is log error!
2021-10-19 09:50:58 - DEBUG - This is log debug!
2021-10-19 09:50:58 - 'gender'
Traceback (most recent call last):
  File "D:/python3Project/test.py", line 18, in <module>
    gender = people_info["gender"]
KeyError: 'gender'
 Copy code 

Logging

In a complete project , Most people will introduce a special logging library , and Python Standard library with you logging It's specially designed for logging ,logging The functions and classes defined in the module implement a flexible event log system for the development of applications and libraries . Logging is provided by the standard library module API The key benefit is that all Python Modules can use this logging function . therefore , Your application log can integrate your own log information with information from third-party modules .

logging Although the module is powerful , But its configuration is also cumbersome , In large projects, you usually need to initialize the log separately 、 Configure log format, etc ,K In daily use, I usually treat logging Do the following encapsulation , So that the log can be saved by day , Retain 15 Day's diary , You can configure whether to output to the console and files , As shown below :

#  Realize the daily split and retention of logs 


import os
import sys
import logging
from logging import handlers


PARENT_DIR = os.path.split(os.path.realpath(__file__))[0]  #  Parent directory 
LOGGING_DIR = os.path.join(PARENT_DIR, "log")              #  Log directory 
LOGGING_NAME = "test"                                      #  Log file name 

LOGGING_TO_FILE = True                                     #  Log output file 
LOGGING_TO_CONSOLE = True                                  #  Log output to the console 

LOGGING_WHEN = 'D'                                         #  Log file segmentation dimension 
LOGGING_INTERVAL = 1                                       #  Less interval  when  after , Automatically rebuild files 
LOGGING_BACKUP_COUNT = 15                                  #  Number of logs reserved ,0  Keep all logs 
LOGGING_LEVEL = logging.DEBUG                              #  The log level 
LOGGING_suffix = "%Y.%m.%d.log"                            #  Old log file name 

#  Log output format 
LOGGING_FORMATTER = "%(levelname)s - %(asctime)s - process:%(process)d - %(filename)s - %(name)s - line:%(lineno)d - %(module)s - %(message)s"


def logging_init():
    if not os.path.exists(LOGGING_DIR):
        os.makedirs(LOGGING_DIR)
    logger = logging.getLogger()
    logger.setLevel(LOGGING_LEVEL)
    formatter = logging.Formatter(LOGGING_FORMATTER)

    if LOGGING_TO_FILE:
        file_handler = handlers.TimedRotatingFileHandler(filename=os.path.join(LOGGING_DIR, LOGGING_NAME), when=LOGGING_WHEN, interval=LOGGING_INTERVAL, backupCount=LOGGING_BACKUP_COUNT)
        file_handler.suffix = LOGGING_suffix
        file_handler.setFormatter(formatter)
        logger.addHandler(file_handler)

    if LOGGING_TO_CONSOLE:
        stream_handler = logging.StreamHandler(sys.stderr)
        stream_handler.setFormatter(formatter)
        logger.addHandler(stream_handler)


def logging_test():
    logging.info("This is log info!")
    logging.warning("This is log warn!")
    logging.error("This is log error!")
    logging.debug("This is log debug!")
    people_info = {"name": "Bob", "age": 20}

    try:
        gender = people_info["gender"]
    except Exception as error:
        logging.exception(error)


if __name__ == "__main__":
    logging_init()
    logging_test()
 Copy code 

Output log :

INFO - 2021-10-19 11:28:10,103 - process:15144 - test.py - root - line:52 - test - This is log info!
WARNING - 2021-10-19 11:28:10,105 - process:15144 - test.py - root - line:53 - test - This is log warn!
ERROR - 2021-10-19 11:28:10,105 - process:15144 - test.py - root - line:54 - test - This is log error!
DEBUG - 2021-10-19 11:28:10,105 - process:15144 - test.py - root - line:55 - test - This is log debug!
ERROR - 2021-10-19 11:28:10,105 - process:15144 - test.py - root - line:61 - test - 'gender'
Traceback (most recent call last):
  File "D:/python3Project/test.py", line 59, in logging_test
    gender = people_info["gender"]
KeyError: 'gender'
 Copy code 

It's like this in the console :

02.png

Of course , If you don't need complex functions , I hope to be concise , Just output the log on the console , You can also simply configure it :

import logging
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
logging.getLogger()
 Copy code 

A more elegant solution :Loguru

about logging modular , Even simple use , You also need to define your own format , Here is a more elegant 、 Efficient 、 Concise third-party modules :loguru, The official introduction is :Loguru is a library which aims to bring enjoyable logging in Python. Loguru For the purpose of Python Bring pleasant logging . Here we quote an official GIF To quickly demonstrate its functionality :

01.gif

install

Loguru Support only Python 3.5 And above , Use pip Can be installed :

pip install loguru
 Copy code 

Open the box

Loguru The main concept is that there is only one :logger

from loguru import logger

logger.info("This is log info!")
logger.warning("This is log warn!")
logger.error("This is log error!")
logger.debug("This is log debug!")
 Copy code 

Console output :

03.png

You can see that you don't need to set it manually ,Loguru Some basic information will be configured in advance , Automatic output time 、 The level of logging 、 Module name 、 Line number and other information , And according to the level , Different colors are also automatically set , Convenient observation , It's really out of the box !

add() / remove()

If you want to customize the log level , Custom log format , What should I do to save the log to a file ? And logging Different modules , Unwanted Handler, Unwanted Formatter, Just one add() Function is OK , For example, we want to save the log to a file :

from loguru import logger

logger.add('test.log')
logger.debug('this is a debug')
 Copy code 

We don't need to look like logging Like a module, declare another FileHandler 了 , Just a line add() Statement handling , After running, you will find the directory test.log There is also the output of the console debug Information .

And add() On the contrary ,remove() Statement can delete the configuration we added :

from loguru import logger

log_file = logger.add('test.log')
logger.debug('This is log debug!')
logger.remove(log_file)
logger.debug('This is another log debug!')
 Copy code 

The console will output two messages debug Information :

2021-10-19 13:53:36.610 | DEBUG    | __main__:<module>:86 - This is log debug!
2021-10-19 13:53:36.611 | DEBUG    | __main__:<module>:88 - This is another log debug!
 Copy code 

and test.log There is only one in the log file debug Information , The reason is that we are in the second debug Statement is preceded by remove() sentence .

Complete parameters

Loguru Very powerful support for output to file configuration , For example, it supports output to multiple files , Output by level , Create new files too large , Too long automatic deletion and so on . Let's take a closer look at add() Detailed parameters of the statement :

Basic grammar :

add(sink, *, level='DEBUG', format='<green>{time:YYYY-MM-DD HH:mm:ss.SSS}</green> | <level>{level: <8}</level> | <cyan>{name}</cyan>:<cyan>{function}</cyan>:<cyan>{line}</cyan> - <level>{message}</level>', filter=None, colorize=None, serialize=False, backtrace=True, diagnose=True, enqueue=False, catch=True, **kwargs)
 Copy code 

Definition of basic parameters :

  • sink: It could be a file object , for example sys.stderr or open('file.log', 'w'), It can also be str String or pathlib.Path object , That is, the file path , It can also be a way , You can define your own output implementation , It can also be a logging Modular Handler, such as FileHandler、StreamHandler etc. , It can also be coroutine function, That is, a function that returns a coroutine object, etc .
  • level: Log output and save level .
  • format: Log format template .
  • filter: An optional instruction , Used to determine whether messages for each record should be sent to sink.
  • colorize: Whether the color mark contained in the formatted message should be converted to a color for terminal coloring ansi Code , Or otherwise . without , According to sink Is it tty( Teletypewriter abbreviation ) Automatic selection .
  • serialize: Sending to sink Before , Whether the recorded message should be converted to JSON character string .
  • backtrace: Whether the formatted exception trace should be extended up , Beyond capture point , To display the full stack trace of the build error .
  • diagnose: Should exception tracking display variable values to simplify debugging . It is recommended to set... In the production environment False, Avoid disclosure of sensitive data .
  • enqueue: Whether the message to be recorded should arrive at sink First, through the multi process security queue , This is useful when logging to files through multiple processes , The benefit of this is to make logging calls non blocking .
  • catch: Whether to automatically capture sink An error occurred while processing log messages , If True, Will be in sys.stderr The exception message is displayed on the , But the exception will not propagate to sink, To prevent the application from crashing .
  • **kwargs: Additional parameters valid only for configuration coroutines or file receivers ( See below ).

If and only if sink When it's a coplanar function , The following parameters apply :

  • loop: The event loop in which the asynchronous logging task is scheduled and executed. . If None, Will use asyncio.get_event_loop() Return loop .

If and only if sink Is the file path , The following parameters apply :

  • rotation: A condition , Indicates when the currently logged file should be closed and a new file started .
  • **retention **: Instructions for filtering old files , Old files are deleted during the cycle or at the end of the program .
  • compression: Log files should be converted to a compressed or archive format when closed .
  • delay: It's configuration sink Create file immediately after , Or is it delayed until the first recorded message . The default is False.
  • mode: built-in open() Function on mode , The default is a( Open file in append mode ).
  • buffering: built-in open() Function buffer strategy , The default is 1( Line buffer file ).
  • encoding: built-in open() The file encoding of the function , If None, The default is locale.getpreferredencoding().
  • **kwargs: Others are passed to the built-in open() The parameters of the function .

With so many parameters, you can see add() The power of functions , Just one function can implement logging Many functions of the module , Next, we introduce some common methods .

rotation Log file separation

add() Functional rotation Parameters , You can create a new log file at a fixed time , For example, set daily 0 Click to create a new log file :

logger.add('runtime_{time}.log', rotation='00:00')
 Copy code 

Set more than 500 MB Create a new log file :

logger.add('runtime_{time}.log', rotation="500 MB")
 Copy code 

Set to create a new... Every other week log file :

logger.add('runtime_{time}.log', rotation='1 week')
 Copy code 

retention Log retention time

add() Functional retention Parameters , You can set the maximum retention time of the log , For example, set the maximum retention time of log files 15 God :

logger.add('runtime_{time}.log', retention='15 days')
 Copy code 

Set the maximum number of log files to be reserved 10 individual :

logger.add('runtime_{time}.log', retention=10)
 Copy code 

It can also be a datetime.timedelta object , For example, set the maximum retention of log files 5 Hours :

import datetime
from loguru import logger

logger.add('runtime_{time}.log', retention=datetime.timedelta(hours=5))
 Copy code 

compression Log compression format

add() Functional compression Parameters , You can configure the compression format of log files , This can save more storage space , For example, setting and using zip File format save :

logger.add('runtime_{time}.log', compression='zip')
 Copy code 

Its format supports :gzbz2xzlzmatartar.gztar.bz2tar.xz

String formatting

Loguru At output log It also provides a very friendly string formatting function , amount to str.format()

logger.info('If you are using Python {}, prefer {feature} of course!', 3.6, feature='f-strings')
 Copy code 

Output :

2021-10-19 14:59:06.412 | INFO     | __main__:<module>:3 - If you are using Python 3.6, prefer f-strings of course!
 Copy code 

Exception tracing

stay Loguru You can directly use the decorator provided by it to directly capture exceptions , And the log obtained is extremely detailed :

from loguru import logger


@logger.catch
def my_function(x, y, z):
    # An error? It's caught anyway!
    return 1 / (x + y + z)


my_function(0, 0, 0)
 Copy code 

Log output :

2021-10-19 15:04:51.675 | ERROR    | __main__:<module>:10 - An error has been caught in function '<module>', process 'MainProcess' (30456), thread 'MainThread' (26268):
Traceback (most recent call last):

> File "D:/python3Project\test.py", line 10, in <module>
    my_function(0, 0, 0)
    └ <function my_function at 0x014CDFA8>

  File "D:/python3Project\test.py", line 7, in my_function
    return 1 / (x + y + z)
                │   │   └ 0
                │   └ 00

ZeroDivisionError: division by zero
 Copy code 

The output on the console is like this :

04.png

comparison Logging,Loguru In terms of configuration 、 The log output style is also exception tracking , Are far better than Logging, Use Loguru It can undoubtedly improve the efficiency of developers . This article only introduces some common methods , For more information, please refer to Loguru Official documents Or pay attention to Loguru GitHub.

copyright notice
author[Brother K reptile],Please bring the original link to reprint, thank you.
https://en.pythonmana.com/2022/01/202201310243486199.html

Random recommended