python～flask

Python标准模块之logging模块

2017-01-24  本文已影响169人  Sank

在写Python程序的时候不论老手或者新手一般都会用print 函数去console中打印信息，可以用来调试，警告，等等，的确很方便。写小程序的时候没有什么问题，当你的程序很庞大的时候，是不是为打印出来的信息过多而烦恼，时不时的要注释掉或者反注释print函数，而且有些信息不仅仅是希望打印在console中，而且也希望记录下log文件中，或者kafka中呢。 其实python 自带的logging 模块就非常的好用，基本可以满足所有需求。

Python logging 模块的官方文档地址为：https://docs.python.org/2/howto/logging.html#logging-basic-tutorial

一个简单的例子

import logging
logging.warning('Watch out!')  # will print a message to the console
logging.info('I told you so')  # will not print anything

如果你输入上面程序，你会看到什么呢？ 下面的信息会打印出来：

WARNING:root:Watch out!

你会发现Info的信息并没有打印出来，这是因为logging 默认的Level是warning，只有在比warning更严重的message才会打印出来。 logging模块中各个Level日志级别关系为：CRITICAL > ERROR > WARNING > INFO > DEBUG > NOTSET，当然也可以自定义。

logging to file

一个简单的例子

import logging
logging.basicConfig(filename='example.log',level=logging.DEBUG)
logging.debug('This message should go to the log file')
logging.info('So should this')
logging.warning('And this, too')

结果就是，产生一个example.log 文件，内容为

DEBUG:root:This message should go to the log file
INFO:root:So should this
WARNING:root:And this, too

Logging from multiple modules

如果你的程序包含很多的模块，那么该如何使用和配置logging 模块呢，看下面的例子

# myapp.py
import logging
import mylib

def main():
    logging.basicConfig(filename='myapp.log', level=logging.INFO)
    logging.info('Started')
    mylib.do_something()
    logging.info('Finished')

if __name__ == '__main__':
    main()

# mylib.py
import logging

def do_something():
    logging.info('Doing something')

如果你跑上面的程序，在myapp.log文件里面的内容为

INFO:root:Started
INFO:root:Doing something
INFO:root:Finished

格式化log

在log中我们经常可以看到时间戳，机器名等信息，这些会使我们的log 更加的专业，python的logging 模块当然也支持这种功能。 看下面的例子

import logging
logging.basicConfig(format='%(asctime)s %(message)s')
logging.warning('is when this event was logged.')

输出结果为

2010-12-12 11:41:42,612 is when this event was logged.

其中时间格式是可以调整的，比如

import logging
logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p')
logging.warning('is when this event was logged.')

输出为

12/12/2010 11:46:36 AM is when this event was logged.

logging.basicConfig 函数中各个参数的说明如下：

上面为基本的简单用法，可以满足大部分的需求

logging高级用法

python的logging其实是由下面四个部分组成:

看个例子

import logging

# create logger
logger = logging.getLogger('simple_example')
logger.setLevel(logging.DEBUG)

# create console handler and set level to debug
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)

# create formatter
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# add formatter to ch
ch.setFormatter(formatter)

# add ch to logger
logger.addHandler(ch)

# 'application' code
logger.debug('debug message')
logger.info('info message')
logger.warn('warn message')
logger.error('error message')
logger.critical('critical message')

输出结果是：

$ python simple_logging_module.py
2005-03-19 15:10:26,618 - simple_example - DEBUG - debug message
2005-03-19 15:10:26,620 - simple_example - INFO - info message
2005-03-19 15:10:26,695 - simple_example - WARNING - warn message
2005-03-19 15:10:26,697 - simple_example - ERROR - error message
2005-03-19 15:10:26,773 - simple_example - CRITICAL - critical message

过程比较清晰，先建logger，handler， formatter。然后给handler指定formatter，再给logger添加handler。 一个logger其实可以添加多个handler，而每个handler的formatter可以不同。这样你就可以既在console 中打印信息，又在log文件中记录信息。

下面这个例子就是这样：

import logging
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s',
datefmt='%a, %d %b %Y %H:%M:%S',
filename='myapp.log',
filemode='w')

#定义一个StreamHandler，将INFO级别或更高的日志信息打印到标准错误，并将其添加到当前的日志处理对象#
console = logging.StreamHandler()
console.setLevel(logging.INFO)
formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
console.setFormatter(formatter)
logging.getLogger('').addHandler(console)

logging.debug('This is debug message')
logging.info('This is info message')
logging.warning('This is warning message')

输出为：

屏幕上打印:
root : INFO This is info message
root : WARNING This is warning message
./myapp.log文件中内容为:
Sun, 24 May 2009 21:48:54 demo2.py[line:11] DEBUG This is debug message
Sun, 24 May 2009 21:48:54 demo2.py[line:12] INFO This is info message
Sun, 24 May 2009 21:48:54 demo2.py[line:13] WARNING This is warning message

回滚日志

你可能不希望日子文件太大，所以你可能需要用到回滚日志的handler，看下面的例子

import logging
from logging.handlers import RotatingFileHandler
#################################################################################################
#定义一个RotatingFileHandler，最多备份5个日志文件，每个日志文件最大10M
Rthandler = RotatingFileHandler('myapp.log', maxBytes=10*1024*1024,backupCount=5)
Rthandler.setLevel(logging.INFO)
formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
Rthandler.setFormatter(formatter)
logging.getLogger('').addHandler(Rthandler)
################################################################################################

python logging 模块提供下面几种handlers

自己创建handler

当然你可以自己创建一个handler去处理，比如kafka， 看下面的例子

from kafka.client import KafkaClient
from kafka.producer import SimpleProducer,KeyedProducer
import logging,sys

class KafkaLoggingHandler(logging.Handler):

    def __init__(self, host, port, topic, key=None):
        logging.Handler.__init__(self)
        self.kafka_client = KafkaClient(host, port)
        self.key = key
        if key is None:
            self.producer = SimpleProducer(self.kafka_client, topic)
        else:
            self.producer = KeyedProducer(self.kafka_client, topic)

    def emit(self, record):
        #drop kafka logging to avoid infinite recursion
        if record.name == 'kafka':
            return
        try:
            #use default formatting
            msg = self.format(record)
            #produce message
            if self.key is None:
                self.producer.send_messages(msg)
            else:
                self.producer.send(self.key, msg)
        except:
            import traceback
            ei = sys.exc_info()
            traceback.print_exception(ei[0], ei[1], ei[2], None, sys.stderr)
            del ei

    def close(self):
        self.producer.stop()
        logging.Handler.close(self)

kh = KafkaLoggingHandler("localhost", 9092, "test_log")
#OR
#kh = KafkaLoggingHandler("localhost", 9092, "test_log", "key1")

logger = logging.getLogger("")
logger.setLevel(logging.DEBUG)
logger.addHandler(kh)
logger.info("The %s boxing wizards jump %s", 5, "quickly")
logger.debug("The quick brown %s jumps over the lazy %s", "fox",  "dog")
try:
    import math
    math.exp(1000)
except:
    logger.exception("Problem with %s", "math.exp")

P.S. The handler uses this Kafka client: https://github.com/mumrah/kafka-python

通过logging.config模块配置日志

logging模块的配置其实可以通过文件来配置，这样更加方便，看下面的例子

#logger.conf
###############################################
[loggers]
keys=root,example01,example02
[logger_root]
level=DEBUG
handlers=hand01,hand02
[logger_example01]
handlers=hand01,hand02
qualname=example01
propagate=0
[logger_example02]
handlers=hand01,hand03
qualname=example02
propagate=0
###############################################
[handlers]
keys=hand01,hand02,hand03
[handler_hand01]
class=StreamHandler
level=INFO
formatter=form02
args=(sys.stderr,)
[handler_hand02]
class=FileHandler
level=DEBUG
formatter=form01
args=('myapp.log', 'a')
[handler_hand03]
class=handlers.RotatingFileHandler
level=INFO
formatter=form02
args=('myapp.log', 'a', 10*1024*1024, 5)
###############################################
[formatters]
keys=form01,form02
[formatter_form01]
format=%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s
datefmt=%a, %d %b %Y %H:%M:%S
[formatter_form02]
format=%(name)-12s: %(levelname)-8s %(message)s
datefmt=

import logging
import logging.config
logging.config.fileConfig("logger.conf")
logger = logging.getLogger("example01")
logger.debug('This is debug message')
logger.info('This is info message')
logger.warning('This is warning message')
上一篇下一篇

猜你喜欢

热点阅读