current position:Home>How to automatically identify n + 1 queries in Django applications (2)?

How to automatically identify n + 1 queries in Django applications (2)?

2022-02-01 15:47:50 DJ_ Tutorial

problem :

  • 1、 Log output in the console
  • 2、 From the output log, we can't determine which interface has the problem

improvement :

  • 1、 Save the log in a file , It is convenient for us to check which interface problems exist regularly
  • 2、 Change the log format , Change the current url Splice in log format , It's easy for us to locate the problem
 Format :
Potential n+1 query detected on `<model>.<field>`
 Output :
2021-11-29 11:20:24,515;WARNING;notifiers.py:40;Potential n+1 query detected on `OutboundOrder.created_by`
 Copy code 

result :

f1e81d2f629d68b91c9801ada9a4fde.png

establish nplusone.py

1、 Redefine the log format

The of the current interface url Spliced in log format

class LazyLoadMessage2(Message2):
    label = 'n_plus_one'
    formatter = '{url}:Potential n+1 query detected on `{model}.{field}`'


class EagerLoadMessage2(Message2):
    label = 'unused_eager_load'
    formatter = '{url}:Potential unnecessary eager load detected on `{model}.{field}`'
 Copy code 

2、 rewrite Message

take url Splice in log

class Message2(Message):
    def __init__(self, model, field, url):
        self.url = url
        super(Message2, self).__init__(model, field)

    @property
    def message(self):
        return self.formatter.format(
            label=self.label,
            model=self.model.__name__,
            field=self.field,
            url=self.url
        )
 Copy code 

3、 Rewrite the listener function

setup Method added request Parameters ,handle_lazy Pass the current interface path to the corresponding Message

class LazyListener2(LazyListener):
    def setup(self, **kwargs):
        self.request = kwargs.get('request', None)
        super(LazyListener2, self).setup()

    def handle_lazy(self, caller, args=None, kwargs=None, context=None, ret=None,
                    parser=None):
        model, instance, field = parser(args, kwargs, context)
        if instance in self.loaded and instance not in self.ignore:
            url = self.request.path if self.request else '' 
            message = LazyLoadMessage2(model, field, url)
            self.parent.notify(message)


class EagerListener2(EagerListener):
    def setup(self, **kwargs):
        self.request = kwargs.get('request', None)
        super(EagerListener2, self).setup()

    def log_eager(self):
        self.tracker.prune([each for each in self.touched if each])
        for model, field in self.tracker.unused:
            url = self.request.path if self.request else ''
            message = EagerLoadMessage2(model, field, url)
            self.parent.notify(message)


listeners2 = {
    'lazy_load': LazyListener2,
    'eager_load': EagerListener2,
}
 Copy code 

4、 Rewrite middleware NPlusOneMiddleware

Here we just need to process_request Methods listeners.listeners Change to what we rewritten above listeners2

class NPlusOneMiddleware2(NPlusOneMiddleware):

    def process_request(self, request):
        self.load_config()
        self.listeners[request] = self.listeners.get(request, {})
        for name, listener_type in six.iteritems(listeners2):
            self.listeners[request][name] = listener_type(self)
            self.listeners[request][name].setup(request=request)
 Copy code 

5、 modify settings

Re import middleware and log output

MIDDLEWARE = (
    'common.nplusone.NPlusOneMiddleware2',
    ...
)

LOGGING = {
    ...
    'handlers': {
        ...
        'nplusone': {
            'level': 'DEBUG',
            'class': 'logging.handlers.TimedRotatingFileHandler',
            'filename': LOG_DIR / 'nplusone.log',
            'formatter': 'standard'
        }
    },
    'loggers': {
        ...
        'nplusone': {
            'handlers': ['nplusone'],
            'level': 'WARN',
        },
    }
}
 Copy code 

 Sweep code _ Search for syndication patterns - Standard color plate .png

copyright notice
author[DJ_ Tutorial],Please bring the original link to reprint, thank you.
https://en.pythonmana.com/2022/02/202202011547476917.html

Random recommended