current position:Home>Django frequency control

Django frequency control

2022-01-31 07:37:17 Waiting for

This is my participation 11 The fourth of the yuegengwen challenge 7 God , Check out the activity details :2021 One last more challenge

One 、 Overview of internal principle of frequency control

django rest framework The basic principle of frequency control in is based on the number of visits and time , Through calculation , Of course, we can also define our own frequency control method . The basic principle is as follows :

Enable frequency ,DRF There will be a dictionary inside to record the visitors IP, And the last few days of the visit ( By configuring ) Time of visits , This ensures that the last element in each list is the earliest time requested by the user , Form the following :

IP1:[ Time of the third request , Second request time , Time of first request ,],
IP2:[ Second request time , Time of first request ,],
 Copy code 

Illustrate with examples , For example, I now configure 5 Can only access... In seconds 2 Time , Each time the request reaches the frequency control, judge the requester first IP Whether it is already in this request dictionary , If exist , In judging user requests 5 Number of requests in seconds , If the number of times is less than or equal to 2, Allow request , If more than 2, The frequency is exceeded , Request not allowed .

On the algorithm of request frequency ( With 5 Access up to two times in seconds as an example ):

1. First delete from the list 5 Request seconds ago , The difference between the current request time and the earliest request time is recorded as t1, if t1 Greater than 5 Then it means that the earliest request in the list has been in 5 Seconds away , Delete the , Continue to judge the penultimate request , until t1 Less than 5.

2. When making sure there are only... In the request list 5 When requested in seconds , Then judge the number of requests ( List length ), If the length is greater than 2, Prove more than 5 Access exceeds... In seconds 2 Time , Not allowed , otherwise , Pass and insert the visit time to the top of the list , As the latest visit time .

Two 、 Basic use

Again , Let's first understand the use of frequency control , Later, we will analyze the source code

1. stay utils Create a new file in the directory ,, Add frequency control so that you can only access... Per minute 5 Time


#!/usr/bin/env python3
#_*_ coding:utf-8 _*_
#Author:wdfrom rest_framework.throttling import SimpleRateThrottle

class VisitThrottle(SimpleRateThrottle):
    """5 Up to three visits in a second """
    scope = "WD"  #settings In the configuration file key, Frequency used to obtain the configuration 
    def get_cache_key(self, request, view):
        return self.get_ident(request)
 Copy code Configure global frequency control in


    # Frequency control configuration "DEFAULT_THROTTLE_CLASSES":['utils.throttle.VisitThrottle'],   # Global configuration ,"DEFAULT_THROTTLE_RATES":{
        'WD':'5/m',         # The rate configuration cannot exceed per minute 5 visit ,WD yes scope Defined value ,
 Copy code

from django.conf.urls import url
from django.contrib import admin
from app01 import views

urlpatterns = [

    url(r'^api/v1/auth', views.AuthView.as_view()),
    url(r'^api/v1/order', views.OrderView.as_view()),
 Copy code

from django.db import models

class UserInfo(models.Model):
    user_type_choice = (
        (1," Ordinary users "),
        (2," members "),
    user_type = models.IntegerField(choices=user_type_choice)
    username = models.CharField(max_length=32,unique=True)
    password = models.CharField(max_length=64)

class UserToken(models.Model):
    user = models.OneToOneField(to=UserInfo)
    token = models.CharField(max_length=64)
 Copy code 

Order view

class OrderView(APIView):
    ''' Check the order '''from utils.permissions import MyPremission
    authentication_classes = [Authentication,]    # Add Certification 
    permission_classes = [MyPremission,]         # Add permission control    
    def get(self,request,*args,**kwargs):
        ret = {'code':1000,'msg':" Your order has been completed ",'data':" Bought one. mac"}
        return JsonResponse(ret,safe=True)
 Copy code 

Use postman Verify the following figure , You can see that the frequency limit has worked .

3、 ... and 、 Frequency control source code analysis

In previous articles, we have analyzed DRF Certification of 、 Permission source code , The same goes for frequency control APIView Of dispatch Methods , Reference notes :


def dispatch(self, request, *args, **kwargs):
        `.dispatch()` is pretty much the same as Django's regular dispatch,
        but with extra hooks for startup, finalize, and exception handling.
        self.args = args
        self.kwargs = kwargs
        # Original request Carry on the processing , Enriched some functions #Request(#     request,#     parsers=self.get_parsers(),#     authenticators=self.get_authenticators(),#     negotiator=self.get_content_negotiator(),#     parser_context=parser_context# )#request( original request,[BasicAuthentications object ,])# Get native request,request._request# Get the object of the authentication class ,request.authticators#1. encapsulation request
        request = self.initialize_request(request, *args, **kwargs)
        self.request = request
        self.headers = self.default_response_headers  # deprecate?try:
            self.initial(request, *args, **kwargs)

            # Get the appropriate handler methodif request.method.lower() in self.http_method_names:
                handler = getattr(self, request.method.lower(),
                handler = self.http_method_not_allowed

            response = handler(request, *args, **kwargs)

        except Exception as exc:
            response = self.handle_exception(exc)

        self.response = self.finalize_response(request, response, *args, **kwargs)
        return self.response
 Copy code 

2. perform inital Method ,initial Method execution check_throttles Then start frequency control

def initial(self, request, *args, **kwargs):
        Runs anything that needs to occur prior to calling the method handler.
        self.format_kwarg = self.get_format_suffix(**kwargs)

        # Perform content negotiation and store the accepted info on the request
        neg = self.perform_content_negotiation(request)
        request.accepted_renderer, request.accepted_media_type = neg

        # Determine the API version, if versioning is in use.
        version, scheme = self.determine_version(request, *args, **kwargs)
        request.version, request.versioning_scheme = version, scheme

        # Ensure that the incoming request is permitted#2. Achieve Authentication         self.perform_authentication(request)
        #3. Authority judgment         self.check_permissions(request)
        #4. Frequency limit 
 Copy code 

3. Here is check_throttles Source code , And certification 、 The permission is the same as the list object , By judgment allow_request Method returns a value to determine whether the frequency passes

def check_throttles(self, request):
        Check if request should be throttled.
        Raises an appropriate exception if the request is throttled.
        """for throttle in self.get_throttles(): # Cyclic frequency control results if not throttle.allow_request(request, self): # Judge one of them allow_requestf Return results ,true Then the frequency passes , Otherwise, it returns the number of seconds to wait to access 
                self.throttled(request, throttle.wait())
 Copy code 

4.get_throttles Method , The frequency control object is generated by list generation , And certification 、 Permission has always been

def get_throttles(self):
        Instantiates and returns the list of throttles that this view uses.
        """return [throttle() for throttle in self.throttle_classes] # List generation generates a list of control frequency objects 
 Copy code 

5.self.throttle_classes Property acquisition

class APIView(View):

    # The following policies may be set at either globally, or per-view.
    renderer_classes = api_settings.DEFAULT_RENDERER_CLASSES
    parser_classes = api_settings.DEFAULT_PARSER_CLASSES
    authentication_classes = api_settings.DEFAULT_AUTHENTICATION_CLASSES
    throttle_classes = api_settings.DEFAULT_THROTTLE_CLASSES     # Global configuration of frequency control 
    permission_classes = api_settings.DEFAULT_PERMISSION_CLASSES
    content_negotiation_class = api_settings.DEFAULT_CONTENT_NEGOTIATION_CLASS
    metadata_class = api_settings.DEFAULT_METADATA_CLASS
    versioning_class = api_settings.DEFAULT_VERSIONING_CLASS
 Copy code 

6. Through the above analysis , We know that the frequency control is through judging the frequency in each class allow_request The return value of the playback method to judge whether the frequency passes , Now let's take a look at the SimpleRateThrottle How to achieve , For the analysis part, please see the notes :

SimpleRateThrottle Class source code :

class SimpleRateThrottle(BaseThrottle):
    A simple cache implementation, that only requires `.get_cache_key()`
    to be overridden.

    The rate (requests / seconds) is set by a `rate` attribute on the View
    class.  The attribute is a string of the form 'number_of_requests/period'.

    Period should be one of: ('s', 'sec', 'm', 'min', 'h', 'hour', 'd', 'day')

    Previous request information used for throttling is stored in the cache.
    cache = default_cache  #  Storage request time , Similar to the big dictionary in the example , What we use here is django The cache of 
    timer = time.time
    cache_format = 'throttle_%(scope)s_%(ident)s'
    scope = None

    def __init__(self):
        if not getattr(self, 'rate', None):
            self.rate = self.get_rate()
        self.num_requests, self.duration = self.parse_rate(self.rate)

    def get_cache_key(self, request, view):
 #  Obtain requested key identification , There must be, or an error will be reported , Here you can rewrite , Use user name 、 Or other acts key, The... Used in the example get_ident Method to get the user IP As key
        Should return a unique cache-key which can be used for throttling.
        Must be overridden.

        May return `None` if the request should not be throttled.
        """raise NotImplementedError('.get_cache_key() must be overridden')

    def get_rate(self):   #  Get the configuration rate of the configuration file 
        Determine the string representation of the allowed request rate.
        """if not getattr(self, 'scope', None):  #  By getting common properties scope To get the configured rate 
            msg = ("You must set either `.scope` or `.rate` for '%s' throttle" %
            raise ImproperlyConfigured(msg)

            return self.THROTTLE_RATES[self.scope]
        except KeyError:
            msg = "No default throttle rate set for '%s' scope" % self.scope
            raise ImproperlyConfigured(msg)

    def parse_rate(self, rate):  #  Format rate 
        Given the request rate string, return a two tuple of:
        <allowed number of requests>, <period of time in seconds>
        """if rate is None:
            return (None, None)
        num, period = rate.split('/')  #  Separate strings 
        num_requests = int(num)
        duration = {'s': 1, 'm': 60, 'h': 3600, 'd': 86400}[period[0]]  #  Convert time to number , Example configuration 5/m,m To 60 second 
        return (num_requests, duration)

    def allow_request(self, request, view):   #   Determine whether the request rate passes 
        Implement the check to see if the request should be throttled.

        On success calls `throttle_success`.
        On failure calls `throttle_failure`.
        """if self.rate is None:
            return True

        self.key = self.get_cache_key(request, view)
        if self.key is None:
            return True

        self.history = self.cache.get(self.key, []) = self.timer()

        # Drop any requests from the history which have now passed the# throttle durationwhile self.history and self.history[-1] <= - self.duration:   #  Implementation principle of frequency judgment , Examples have been given to illustrate 
        if len(self.history) >= self.num_requests:
            return self.throttle_failure()
        return self.throttle_success()

    def throttle_success(self):       #  The frequency is returned by true
        Inserts the current request's timestamp along with the key
        into the cache.
        self.cache.set(self.key, self.history, self.duration)
        return True

    def throttle_failure(self):    #  Not by returning false
        Called when a request to the API has failed due to throttling.
        """return False

    def wait(self):               #  Return wait time 
        Returns the recommended next request time in seconds.
        """if self.history:
            remaining_duration = self.duration - ( - self.history[-1])
            remaining_duration = self.duration

        available_requests = self.num_requests - len(self.history) + 1
        if available_requests <= 0:
            return None

        return remaining_duration / float(available_requests)
 Copy code 

get_ident Method source code , This method is used to get the requested IP:

def get_ident(self, request):
        Identify the machine making the request by parsing HTTP_X_FORWARDED_FOR
        if present and number of proxies is > 0. If not use all of
        HTTP_X_FORWARDED_FOR if it is available, if not use REMOTE_ADDR.
        xff = request.META.get('HTTP_X_FORWARDED_FOR')
        remote_addr = request.META.get('REMOTE_ADDR')
        # here request It's after encapsulation requst,django The original is request._request.META  You can also get 
        num_proxies = api_settings.NUM_PROXIES

        if num_proxies is not None:
            if num_proxies == 0 or xff is None:
                return remote_addr
            addrs = xff.split(',')
            client_addr = addrs[-min(num_proxies, len(addrs))]
            return client_addr.strip()

        return ''.join(xff.split()) if xff else remote_addr
 Copy code 

copyright notice
author[Waiting for],Please bring the original link to reprint, thank you.

Random recommended