current position:Home>Summary of 22 advanced Python knowledge points, dry goods!

Summary of 22 advanced Python knowledge points, dry goods!

2022-02-02 04:44:04 Mssyaa

No.1 Everything is the object

as everyone knows ,Java The emphasis on “ Everything is the object ”, however Python Object oriented ratio in Java More thorough , because Python Class in (class) Also object , function (function) Also object , and Python Your code and modules are also objects .

  • Python Functions and classes in can be assigned to a variable
  • Python Functions and classes in can be stored in collection objects
  • Python In, functions and classes can be passed to functions as parameters of a function
  • Python Functions and classes in can be used as return values

Step.1

3.png

Step.2

4.png

Step.3

5.png

Step.4

6.png

No.2 keyword type、object、class The relationship between

stay Python in ,object Examples are type,object It's the top class , No base class ;type Examples are type,type The base class of is object.Python The base class of the built-in type in is object, But they are all made up of type Instantiate , The specific value is instantiated from the built-in type . stay Python2.x If the user-defined class in the syntax of does not specify the base class explicitly, there is no base class by default , stay Python3.x In the grammar of , Specify the base class as object.

7.png

No.3 Python Built in type

stay Python in , Objects have 3 A feature attribute :

  • Address in memory , Use id() Function to view
  • Type of object
  • Object's default value

Step.1 None type

stay Python When the interpreter starts , Will create a None Type of None object , also None There is only one global object .

Step.2 value type

  • ini type
  • float type
  • complex type
  • bool type

Step.3 Iteration type

stay Python in , Iteration types can use loops to traverse .

Step.4 Sequence type

  • list
  • tuple
  • str
  • array
  • range
  • bytes, bytearray, memoryvie( Binary sequence )

Step.5 Mapping type

  • dict

Step.6 Collection types

  • set
  • frozenset

Step.7 Context management type

  • with sentence

Step.8 Other types

  • modular
  • class
  • example
  • function
  • Method
  • Code
  • object object
  • type object
  • ellipsis( Ellipsis )
  • notimplemented

NO.4 Magical function

Python The magic function in starts with a double underscore , Ends with double underscores

No.5 Duck type and white goose type

Duck type is the inference style in programming , In duck types, focus on how objects are used, not the type itself . Duck types work like polymorphisms but do not inherit . The concept of duck type comes from :“ When you see a bird walk like a duck 、 Swimming like a duck 、 It also sounds like a duck , Then this bird can be called a duck .”

8.png The white goose type means as long as  cls  Is an abstract base class , namely  cls  The metaclass of is  abc.ABCMeta , You can use  isinstance(obj, cls) .

No.6 agreement 、 Abstract base class 、abc Inheritance relationship between modules and sequences

  • agreement :Python Informal interface in , Is allowed Python The way to achieve polymorphism , The agreement is informal , There is no compulsion , Defined by conventions and documents .
  • Interface : It generally refers to an abstraction provided by an entity to the outside world ( Can be another entity ), To separate external communication methods from internal operation , So that it can be modified internally without affecting the way other external entities interact with it .

We can use Monkey patch To implement the protocol , So what is a monkey patch ?

Monkey patch is to modify modules or classes at run time , Don't modify the source code , So as to realize the target protocol interface operation , This is the so-called monkey patch .

Tips: The name monkey patch originated from Zope frame , Developers are modifying Zope Of Bug when , Often add updated parts to the back of the program , these Miscellaneous Army Patch My English name is guerilla patch, Later it was written as gorllia, Then it became monkey.

The main function of monkey patch is :

  • Replace methods at run time 、 attribute
  • Add functions to the program itself without modifying the source code
  • Add a patch to the runtime object , Not on the source code on disk

The application case : Suppose you write a big project , Used everywhere json Module to parse json file , But it turns out ujson Than json Higher performance , Modifying the source code requires many changes , So just add... At the program entry :

9.png

Python Abstract base classes have an important practical advantage : have access to  register  Class methods put a class in the end user's code “ Statement ” For an abstract base class “ fictitious ” Son class ( So , The registered class must meet the requirements of abstracting its class for method name and signature , The most important thing is to meet the bottom Layer semantic contract ; however , You don't need to understand the abstract base class when developing that class , Let alone inherit the abstract base class . Sometimes , In order for abstract classes to recognize subclasses , You don't even have to register . To curb the urge to create abstract base classes . Abusing abstract base classes can have disastrous consequences , It shows that language pays too much attention to surface form .

  • Abstract base classes cannot be instantiated ( Cannot create object ), It is usually used as a base class for subclasses to inherit , Override virtual functions in subclasses , Implement specific interfaces .
  • Determine the type of an object
  • Force subclasses to implement certain methods

Definition and use of abstract base classes

10.png It is worth noting that :Python 3.0-Python3.3 Between , The syntax for inheriting an abstract base class is class ClassName(metaclass=adc.ABCMeta), Other versions are :class ClassName(abc.ABC).

  • collections.abc Of each abstract base class in the module UML Class diagram

12.jpg

No.7 isinstence and type The difference between

13.png

python There's me Full of vitality Dry cargo is full.

No.8 Class variables and instance variables

  • Instance variables can only be called through instances of classes
  • Modify the properties of the object created by the template object , The properties of the template object will not change
  • Modify the properties of the template object , The properties of the object created by the template object will change

14.png

No.9 The search order of class and instance properties and methods

  • stay Python 2.2 Before, there were only classic classes , To Python2.7 Also compatible with classic classes ,Python3.x Only new classes will be used in the future ,Python Previous versions will also be compatible with new classes
  • Python 2.2 And previous classes have no base class ,Python New classes need to explicitly inherit from object, Even if you do not explicitly inherit, you will inherit from by default object
  • Classical classes adopt the matching method of depth first principle from left to right when class multiple inheritance . And the new class adopts C3 Algorithm
  • The classic class doesn't have MRO and instance.mro() Called

The following inheritance relationships are assumed: :

image.png

use DFS( Depth first search algorithm ) When calling the A Of say_hello() Method time , The system will go to B Find if B There's no such thing as , Then go D Search for , Obviously D This method exists in , however DFS There will be defects in the following inheritance relationships :

16.png

stay A Call in instance object say_hello When the method is used , The system will go first B Search for , because B There is no definition of this method in the class , So I will go to D Search for ,D Not in class , The system will think that the method is not defined , In fact, this method is C It defines . So consider using BFS( Breadth first search algorithm ), So the question goes back to the first inheritance relationship , Assume C and D Have duplicate name method , Calling A The method of an instance of , It should be in B Search for , Should call D The method in , But use BFS When ,C Methods in the class will override D Methods in class . stay Python 2.3 In later versions , Use C3 Algorithm :

17.png

Use C3 The second inheritance order after the algorithm :

18.png

Use C3 The first inheritance order after the algorithm :

19.png

Here we only introduce the function and evolution history of the algorithm

No.10 Class method 、 Instance methods and static methods

20.png

Instance methods can only be called through instances of classes ; Static methods are independent 、 Stateless functions , Closely depend on the namespace of the class ; Class methods are used to obtain the data maintained in the class , such as :

21.png

No.11 Data encapsulation and private properties

Python Use double underline in + The implementation of attribute names is similar to that in static languages private Decoration to realize data encapsulation .

22.png

No.12 Python The introspection mechanism of

introspection (introspection) It's a self checking behavior . In computer programming , Introspection refers to this ability : Examine something to determine what it is 、 What it knows and what it can do . Introspection provides programmers with great flexibility and control .

  • dir([obj]): Returns a sorted list of the property names of any object passed to it ( There will be some special properties that are not included )
  • getattr(obj, attr): Returns any property of any object , Calling this method will return obj Middle name is attr The value of the property of value
  • ... ...

No.13 super function

Python3.x and Python2.x One difference is : Python 3 It can be used directly super().xxx Instead of super(type[, object-or-type]).xxx .

super() Function to call MRO( Class method parsing sequence table ) Method of the next class of .

No.14 Mixin Inherit

In design Mixin Class is mixed in as a function and inherited from Mixin Class . Use Mixin Class implementation of multiple inheritance should pay attention to :

  • Mixin Class must represent a function
  • Single responsibility , If you want to have multiple functions , You have to design multiple Mixin class
  • Implementation independent of subclasses ,Mixin The existence of classes only adds the functional features of subclasses
  • Even if the subclass does not inherit this Mixin Classes can also work

23.png

python There's me Full of vitality Dry cargo is full.

No.15 Context manager with Statements and contextlib simplify

Common exception capture mechanism :

24.png with Simplified exception capture writing :

25.png

26.png

No.16 Classification of sequence types

  • Container sequence :list tuple deque
  • Flat sequence :str bytes bytearray array.array
  • Variable sequence :list deque bytearray array
  • Immutable sequence :str tuple bytes

No.17 +、+=、extend() The difference between is different from the application scenario

Let's start with test cases :

27.png It can be seen from the source code fragment :

28.png

No.18 Use bisect Maintain a sorted sequence

29.png

No.19 deque Class explanation

deque yes Python A double ended queue in , At both ends of the queue O ( 1 ) O(1) Efficiency of inserting data , be located collections Module .

30.png
deque Class source code :

31.png

32.png

33.png

34.png

35.png

36.png

deque

list When storing data , The internal implementation is an array , The search speed of array is very fast , But the speed of inserting and deleting data is worrying .deque The internal implementation of double ended list is double ended queue .deuque Applicable to queues and stacks , And it's thread safe .

deque Provide append() and pop() Function implementation in deque Adding and popping data at the end , Provide appendleft() and popleft() Function implementation in deque Header add and pop-up elements . this 4 The time complexity of each function is O ( 1 ) O(1) Of , however list The time complexity is up to O ( n ) O(n) .

establish deque queue

37.png

Source code

38.png

39.png

40.png

41.png

42.png

43.png

44.png

ChainMap

Used to merge multiple dictionaries .

The application case

45.png

Source code

46.png

47.png

48.png

UserDict

UserDict yes MutableMapping and Mapping Subclasses of , It inherited MutableMapping.update and Mapping.get Two important ways .

The application case

49.png

Source code

50.png

51.png

No.20 Python Variables in and garbage collection mechanism

Python And Java The variables are essentially different ,Python The variable ability of is a pointer . When Python The interpreter performs number=1 When , Actually, first create a... In memory int object , And then number Point to this int The memory address of the object , Also is to number“ stick ” stay int On the object , The test cases are as follows :

52.png

== and is The difference is that the former judges whether the values are equal , The latter judges the object id Whether the values are equal .

53.png Python There is an optimization mechanism called intern, Small integers often used like this 、 Little string , It is created at run time , And the overall situation is unique .

Python Medium del Statements are not equivalent to C++ Medium delete,Python Medium del Is to delete the point of this object , When the object has no point ,Python The virtual machine will delete this object .

No.21 Python Metaclass programming

property Dynamic properties

54.png stay Python in , Add... To the function @property Decorators can make functions access... Like variables .

__getattr__ and __getattribute__ Use of functions

__getattr__ When looking for attributes , This function is called if the property cannot be found .

55.png __getattribute__ This method is called before calling the property .

56.png

property descriptor

Implement... In a class __get__()__set__() and __delete__() Are all attribute descriptors .

Data property descriptor

57.png

Non data attribute descriptor

stay Python In the new category of , Access to object properties will call __getattribute__() Method , It allows us to customize the access behavior when accessing objects , It's worth noting that beware of infinite recursion .__getattriubte__() It is the entrance to all methods and properties , When this method is called, it will be executed according to certain rules __dict__ Find the corresponding attribute value or object in , If it is not found, it will be called __getattr__() Method , Corresponding __setattr__() and __delattr__() Methods are used to customize the assignment behavior of an attribute and the behavior of deleting an attribute . The concept of descriptor is Python 2.2 Introduce ,__get__()__set__()__delete__() Define and take out respectively 、 Set up 、 The act of deleting the value of a descriptor .

  • It is worth noting that , As long as you implement any of these three methods, it is a descriptor .
  • Only realize __get__() Methods are called non data descriptors , Can only be read after initialization .
  • At the same time to realize __get__() and __set__() Methods are called data descriptors , Properties are readable and writable .

Priority rule for property access

The properties of objects are generally __dict__ Storage in , stay Python in ,__getattribute__() Implement the relevant rules of attribute access .

Suppose there are instances obj, attribute number stay obj The search process in is like this :

  • Search the base class list type(b).__mro__, Until the property is found , And assign it to descr.
  • Judge descr The type of , If it is a data descriptor, call descr.__get__(b, type(b)), And return the result .
  • If it's something else ( Non-data descriptor 、 General attribute 、 Type not found ) Then find the instance obj Instance properties for , That is to say obj.__dict__.
  • If in obj.__dict__ No related properties found , Will return to descr Judgement on .
  • If you judge again descr Type is non data descriptor , Will call descr.__get__(b, type(b)), And return the result , End to perform .
  • If descr It's a common property , Just return the result directly .
  • If you don't find it the second time , It's empty , It will trigger AttributeError abnormal , And end the search .

Use a flow chart to show :

58.png

__new__() and __init__() The difference between

  • __new__() Function is used to control the generation process of the object , Call before generation on object .
  • __init__() Function is used to perfect the object , After object creation .
  • If __new__() The function does not return an object , Will not call __init__() function .

Custom metaclasses

stay Python Everything is an object , Class is used to describe how to generate objects , stay Python Middle class is also an object , The reason is that it has the ability to create objects . When Python The interpreter executes to class At the time of statement , Will create this so-called class object . Since a class is an object , Then you can dynamically create classes . Here we use type() function , The following is the constructor source code of this function :

59.png Thus we can see that ,type() Receive the amount description of a class and return a class .

60.png Metaclasses are used to create classes , Because fatigue is also an object .type() The reason why classes can be created is because tyep() Is a metaclass ,Python All classes in are created by it . stay Python in , We can use the of an object __class__ Property to determine which class this object is generated from , When Python When creating an object of a class ,Python It will be found in this class __metaclass__ attribute . If you find it , Just use it to create objects , If not found , Go to the parent class to find , If there is still no , Go to the module to find , If you haven't found it all the way down , Just use type() establish . To create a metaclass, you can use the following writing :

61.png

Use metaclasses to create API

The main purpose of metaclasses is to create API, such as Python Medium ORM frame .

Python Leader Tim Peters :

“ Metaclasses are deep magic ,99% Users should not have to worry about it at all . If you want to figure out whether you need to use metaclasses or not , Then you don't need it . Those who actually use metaclasses know exactly what they need to do, right , And you don't even have to explain why you use metaclasses, right .”

No.22 Iterators and generators

When there are many elements in the container , It is impossible to read all of them into memory , Then we need an algorithm to calculate the next element , This eliminates the need to create large containers , That's what generators do .

Python Generator use in yield Return value , Every time you call yield Will suspend , Therefore, the generator will not execute all at once , It is calculated when the result is needed , When the function executes to yield When , Will return the value and save the current execution status , That is, the function is suspended . We can use next() Functions and send() Function recovery generator , Deduce the list into [] Switch to () It becomes a generator :

62.png

It is worth noting that , We don't usually use next() Method to get the element , But use for loop . When using while loop , Need to capture StopIteration The generation of anomalies .

Python There is a call stack of stack frames in the virtual machine , The stack frame holds the information and context of the specified code , Each stack frame has its own data stack and block stack , Because these stack frames are stored in heap memory , Make the interpreter have the ability to interrupt and recover stack frames :

63.png This is also the foundation of the generator . As long as we get the generator object anywhere , You can start or pause the generator , Because stack frames exist independently of the caller , This is also the theoretical basis of collaborative process .

Iterators are different from for A way of accessing elements in a collection in a loop , Generally used to traverse data , Iterators provide a lazy way to access data .

have access to for There are several types of loops :

  • Collection data type
  • generator , Including generators and with yield The generator function of

These can be directly for The object called in a loop is called an iteratable object , have access to isinstance() Judge whether an object can be Iterable object . The collection data type is as follows listdictstr Is such as Iterable But it's not Iterator, Can pass iter() Function to get a Iterator object .send() and next() The difference is send() Parameters can be passed to yield() expression , At this time, the parameters passed will be used as yield Value of expression , and yield Is the value returned to the caller , in other words send You can force the last yield Expression value .

python There's me Full of vitality Dry cargo is full.

copyright notice
author[Mssyaa],Please bring the original link to reprint, thank you.
https://en.pythonmana.com/2022/02/202202020443258294.html

Random recommended