current position:Home>New features of python3.6, 3.7, 3.8 and 3.9

New features of python3.6, 3.7, 3.8 and 3.9

2022-06-24 05:12:39Good pie notes

This article lists Python3.6、3.7、3.8、3.9 New features in four versions , Learning them will help to improve the understanding of Python Understanding , Keep up with the latest trends .

One .Python3.6 New characteristics

1. New format string method

New format string method , That is, add... Before the normal string f or F Prefix , The effect is similar to str.format(). such as

name = "red"print(f"He said his name is {name}.") # 'He said his name is red.'

amount to :

print("He said his name is {name}.".format(**locals()))

Besides , This feature also supports nested fields , such as :

import decimalwidth = 10precision = 4value = decimal.Decimal("12.34567")print(f"result: {value:{width}.{precision}}") #'result:  12.35'
2. Variable declaration syntax

You can declare a variable and specify the type as follows :

from typing import List, Dictprimes: List[int] = []captain: str  #  There is no initial value at this time class Starship:  stats: Dict[str, int] = {}
3. Underline numbers

Allow underscores in numbers , To improve the readability of multi digit numbers .

a = 1_000_000_000_000_000    # 1000000000000000b = 0x_FF_FF_FF_FF       # 4294967295

besides , String formatting is also supported _ Options , To print out a more readable numeric string :

'{:_}'.format(1000000)     # '1_000_000''{:_x}'.format(0xFFFFFFFF)   # 'ffff_ffff'
4. Asynchronous generator

stay Python3.5 in , A new grammar has been introduced async and await To implement collaborative programs . But there is a limit , Cannot use... In the same function body at the same time yield and await.Python3.6 in , This restriction has been lifted , Allows you to define asynchronous generators :

async def ticker(delay, to):"""Yield numbers from 0 to *to* every *delay* seconds."""  for i in range(to):    yield i    await asyncio.sleep(delay)
5. Asynchronous parser

Allow in the list list、 aggregate set And the dictionary dict In the parser async or await grammar .

result = [i async for i in aiter() if i % 2]result = [await fun() for fun in funcs if await condition()]
6. Newly added module

Standard library (The Standard Library) A new module has been added in :secrets. This module is used to generate some random numbers with higher security , Used to manage passwords, account authentication, security tokens, as well as related secrets Data such as .

7. Other new features
  • new PYTHONMALLOC Environment variables allow developers to set memory allocators , And registration debug Hook, etc .
  • asyncio The module is more stable 、 Efficient , And is no longer a temporary module , Among them API They are all stable versions .
  • typing The module has also been improved , And is no longer a temporary module .
  • datetime.strftime and date.strftime Start supporting ISO 8601 Time identifier of %G, %u, %V.
  • hashlib and ssl Modules begin to support OpenSSL1.1.0.
  • hashlib Modules begin to support new hash Algorithm , such as BLAKE2, SHA-3 and SHAKE.
  • Windows Upper filesystem and console The default encoding is changed to UTF-8.
  • json Module json.load() and json.loads() Functions begin to support binary Type of input .

Two .Python3.7 New characteristics

Python 3.7 On 2018 year 6 month 27 Promulgated by the , Contains many new features and optimizations , Added many new classes , It can be used for data processing 、 Optimization for Script Compilation and garbage collection and faster asynchrony I/O, Mainly as follows :

  • Reduce the number of data classes in the boilerplate code when processing data with classes .
  • One change that may not be backward compatible involves handling exceptions in the generator .
  • Interpreter oriented “ Development mode ”.
  • Time object with nanosecond resolution .
  • Used by default in the environment UTF-8 Coded UTF-8 Pattern .
  • A new built-in function that triggers the debugger .
1. New built-in functions breakpoint()

Use this built-in function , It is equivalent to setting breakpoints through code , Will automatically enter Pbd Debug mode .

If... Is set in the environment variable PYTHONBREAKPOINT=0 This function is ignored . also ,pdb Just one of many available debuggers , You can set up a new PYTHONBREAKPOINT Environment variables to configure the debugger you want to use .

Here is a simple example , The user needs to enter a number , Determine whether it is the same as the target number :

""" Guess the number game """def guess(target):    user_guess = input(" Please enter the number you guessed  >>> ")    if user_guess == target:        return " You guessed it !"    else:        return " Wrong guess. "if __name__ == '__main__':    a = 100    print(guess(a))

Unfortunately , Even if the number of guesses is the same as the target number , The result of printing is also ‘ Wrong guess. ’, And there are no exceptions or error messages .

To find out what happened , We can insert a breakpoint , Let's debug . In the past, we usually passed print Dafa or IDE Debugging tools for , But now we can use breakpoint().

""" Guess the number game """def guess(target):    user_guess = input(" Please enter the number you guessed  >>> ")    breakpoint()   // Join the line     if user_guess == target:        return " You guessed it !"    else:        return " Wrong guess. "if __name__ == '__main__':    a = 100    print(guess(a))

stay pdb At the prompt , We can call locals() To view all variables of the current local scope .(pdb There are a lot of orders , You can also run it normally Python sentence )

 Please enter the number you guessed  >>> 100> c:\moonrong\py3_test\> if user_guess == target:(Pdb) locals(){'target': 100, 'user_guess': '100'}(Pdb) type(user_guess)<class 'str'>

I see ,target It's an integer , and user_guess Is a string , There is a type comparison error .

2. Types and annotations

from Python 3.5 Start , Type annotations are becoming more and more popular . For those unfamiliar with type cues , This is a completely optional way to annotate code , To specify the type of the variable .

What is annotation ? They are syntax support for associating metadata with variables , It could be any expression , At run time is Python Calculated but ignored . Annotations can be any valid Python expression .

Here is an example of comparison :

#  Without type annotation def foo(bar, baz):#  With type annotation def foo(bar: 'Describe the bar', baz: print('random')) -> 'return thingy':

The above method , It's actually Python The reinforcement of its own weakly typed language , Hope to obtain certain type reliability and robustness , towards Java And so on .

stay Python 3.5 in , The syntax of annotations is standardized , thereafter ,Python Annotation type hints are widely used in the community .

however , Annotations are just a development tool , have access to PyCharm etc. IDE or Mypy Wait for third-party tools to check , It is not a grammatical restriction .

Our previous guessing program adds type annotations , It should be like this :

""" Guess the number game """def guess(target:str):    user_guess:str = input(" Please enter the number you guessed  >>> ")    breakpoint()    if user_guess == target:        return " You guessed it !"    else:        return " Wrong guess. "if __name__ == '__main__':    a:int = 100    print(guess(a))

PyCharm Will give us a gray specification error warning , But will not give red syntax error prompt .

When using annotations as type prompts , There are two main problems : Start performance and forward reference .

  • Evaluating a large number of arbitrary expressions at definition time has a considerable impact on startup performance , and typing Modules are very slow
  • You cannot annotate with undeclared types

typing Part of the reason modules are so slow is , The initial design goal was to not modify the core CPython Interpreter implementation typing modular . As type hints become more and more popular , This restriction has been removed , This means that now there is a right typing Core support for .

And for forward references , See the following example :

class User:    def __init__(self, name: str, prev_user: User) -> None:        pass

The mistake is User The type has not been declared yet , At this time prev_user It can't be defined as User type .

To solve this problem ,Python3.7 Deferred evaluation of comments . also , This change is backwards incompatible , You need to import annotations, Only to Python 4.0 Will become the default behavior .

from __future__ import annotationsclass User:     def __init__(self, name: str, prev_user: User) -> None:        pass

Or as in the following example :

class C:    def validate_b(self, obj: B) -> bool:        ...class B:    ...
3. newly added dataclasses modular

This feature may be Python3.7 More commonly used in the future , What does it do ?

Suppose we need to write the following class :

from datetime import datetimeimport dateutilclass Article(object):    def __init__(self, _id, author_id, title, text, tags=None,        ,    self._id = _id    self.author_id = author_id    self.title = title    self.text = text    self.tags = list() if tags is None else tags    self.created = created    self.edited = edited    if type(self.created) is str:       self.created = dateutil.parser.parse(self.created)    if type(self.edited) is str:       self.edited = dateutil.parser.parse(self.edited)    def __eq__(self, other):        if not isinstance(other, self.__class__):            return NotImplemented        return (self._id, self.author_id) == (other._id, other.author_id)    def __lt__(self, other):        if not isinstance(other, self.__class__):            return NotImplemented        return (self._id, self.author_id) < (other._id, other.author_id)    def __repr__(self):        return '{}(id={}, author_id={}, title={})'.format(                self.__class__.__name__, self._id, self.author_id, self.title)

A large number of initialization attributes need to define default values , You may need to rewrite a bunch of magic methods , To print class instances 、 Compare 、 Sorting and de duplication .

If you use dataclasses To transform , It can be written like this :

from dataclasses import dataclass, fieldfrom typing import Listfrom datetime import datetimeimport [email protected](order=True)   // Note that there class Article(object):    _id: int    author_id: int    title: str = field(compare=False)    text: str = field(repr=False, compare=False)    tags: List[str] = field(default=list(), repr=False, compare=False)    created: datetime = field(, repr=False, compare=False)    edited: datetime = field(, repr=False, compare=False)    def __post_init__(self):       if type(self.created) is str:           self.created = dateutil.parser.parse(self.created)       if type(self.edited) is str:           self.edited = dateutil.parser.parse(self.edited)

This makes classes not only easy to set up , And when we create an instance and print it out , It can also automatically generate beautiful strings . When comparing with other class instances , It will also have appropriate behavior . This is because dataclasses In addition to helping us automatically generate  _init_  Outside method , Some other special methods have also been generated , Such as repr、eq and hash etc. .

Dataclasses Use fields field To provide default values , Manually construct a field() Function can access other options , This changes the default value . for example , There will be field Medium default_factory Set to a lambda function , This function prompts the user for its name .

from dataclasses import dataclass, fieldclass User:    name: str = field(default_factory=lambda: input("enter name"))
4. Generator exception handling

stay Python 3.7 in , The generator raises StopIteration After abnormality ,StopIteration The exception will be converted to RuntimeError abnormal , That way, it won't quietly affect the stack framework of the application . This means that some programs that are not very sensitive about how to handle the behavior of the generator will Python 3.7 Throw out RuntimeError. stay Python 3.6 in , This behavior generates a deprecation warning ; stay Python 3.7 in , It will generate a complete error .

An easy way is to use try/except Code segment , stay StopIteration Propagate to the outside of the generator to capture it . A better solution is to rethink how to build the generator ―― for instance , Use return Statement to terminate the generator , Instead of manually triggering StopIteration.

5. Development mode

Python The interpreter adds a new command line switch :-X, Allows developers to set many low-level options for the interpreter .

This run-time checking mechanism usually has a significant impact on performance , But it is very useful for developers during debugging .

-X The active options include :

  • asyncio Debug mode of the module . This provides more detailed logging and exception handling for asynchronous operations , Abnormal operations can be difficult to debug or reason .
  • Debug hook for memory allocator . This is for writing CPython People who use extensions are very useful . It enables more explicit runtime checks , understand CPython How to allocate and free memory internally .
  • Enable faulthandler modular , After that crash ,traceback Always dump out .
6. High precision time function

Python 3.7 A new class of time functions in returns nanosecond precision time values . Even though Python It's an interpretative language , however Python Victor, the core developer of • Steiner (Victor Stinner) Claim to report time with nanosecond accuracy . The main reason is , In the process of converting other programs ( Like databases ) Recorded time value , It can avoid losing accuracy .

The new time function uses the suffix ns. for instance ,time.processtime() The nanosecond version of is time.processtimens(). Please note that , Not all time functions have corresponding nanosecond versions .

7. Other new features
  • The dictionary now maintains the insertion order . This is in 3.6 Chinese is informal , But now it has become the official language standard . in the majority of cases , ordinary dict Can replace collections.OrderedDict.
  • .pyc The document is deterministic , Supports repeatable builds —— in other words , Always generate the same... For the same input file byte-for-byte Output .
  • newly added contextvars modular , Provide context variables for asynchronous tasks .
  • main The code in displays a deprecation warning (DeprecationWarning).
  • newly added UTF-8 Pattern . stay Linux/Unix System , The... Of the system will be ignored locale, Use UTF-8 As default encoding . In Africa Linux/Unix System , Need to use -X utf8 Option enable UTF-8 Pattern .
  • Allow module definitions getattr、dir function , For deprecating warnings 、 Delay import Sub modules and so on .
  • New threads are stored locally C Language API.
  • to update Unicode Data to 11.0.

3、 ... and .Python3.8 New characteristics

Python3.8 Version on 2019 year 10 month 14 Promulgated by the , Here are Python 3.8 comparison 3.7 New features .

1. Walrus assignment expression

New syntax :=, Assign a value to a variable in a larger expression . It is affectionately called “ Walrus operators ”(walrus operator), Because it looks like walrus' eyes and ivory .

“ Walrus operators ” At some point you can make your code cleaner , such as :

In the following example , Assignment expressions can avoid calling len () two :

if (n := len(a)) > 10:      print(f"List is too long ({n} elements, expected <= 10)")

A similar benefit can be seen in the case that regular expression matching requires the use of matching objects twice , A test is used to determine whether a match has occurred , The other is used to extract sub groups :

discount = 0.0  if (mo :='(\d+)% discount', advertisement)):        discount = float( / 100.0

This operator can also be used in conjunction with while Loop through a value , To detect whether the loop is terminated , And the same value is used again in the loop body :

# Loop over fixed length blocks while (block := != '':     process(block)

Or in a list derivation , Calculate a value in the filter criteria , The same value needs to be used in the expression :

[clean_name.title() for name in names    if (clean_name := normalize('NFC', name)) in allowed_names]

Try to limit the use of walrus operators to clear situations , To reduce complexity and improve readability .

2. Position parameters only

Add a function parameter syntax / Used to indicate that some function parameters must be in the form of position only parameters instead of keyword parameters .

This kind of markup syntax is similar to that of help () Use shown Larry Hastings Of Argument Clinic Tool marked C The same function .

In the following example , Shape parameter a and b Is a positional parameter only ,c or d It can be a location parameter or a keyword parameter , and e or f Keyword parameter required :

def f(a, b, /, c, d, *, e, f):      print(a, b, c, d, e, f)

Here are the legal calls :

f(10, 20, 30, d=40, e=50, f=60)

however , The following are illegal calls :

f(10, b=20, c=30, d=40, e=50, f=60)   # b  Cannot be a keyword parameter f(10, 20, 30, 40, 50, f=60)           # e  Must be a keyword parameter 

One use case for this notation is that it allows pure Python Function to completely simulate the existing C The behavior of a function written in code . for example , Built in pow () Function does not accept keyword arguments :

def pow(x, y, z=None, /):       "Emulate the built in pow() function"       r = x ** y      return r if z is None else r%z

Another use case is to exclude keyword parameters when formal parameter names are not required . for example , Built in len () The signature of the function is len (obj, /). This can rule out this clumsy form of invocation :

len(obj='hello')  # The "obj" keyword argument impairs readability

Another benefit is that marking a parameter as location only will allow the parameter name to be modified in the future without breaking the customer's code . for example , stay statistics Module , The name of the parameter dist May be modified in the future . This makes the following function description possible :

def quantiles(dist, /, *, n=4, method='exclusive')      ...

Because in / The formal parameters on the left are not exposed as available keywords , Other parameter names can still be found in **kwargs Use in :

>>> def f(a, b, /, **kwargs):   ...     print(a, b, kwargs) ... >>> f(10, 20, a=1, b=2, c=3)         # a and b are used in two ways 10 20 {'a': 1, 'b': 2, 'c': 3}

This greatly simplifies the implementation of functions and methods that need to accept arbitrary keyword parameters . for example , Here is collections Code excerpt from module :

class Counter(dict):        def __init__(self, iterable=None, /, **kwds):           # Note "iterable" is a possible keyword argument
3.f String support =

increase = A specifier is used for f-string. In the form of f'{expr=}' Of f The string represents the extension as expression text , Add an equal sign , Plus the evaluation result of the expression . for example :

>>> user = 'eric_idle'  >>> member_since = date(1975, 7, 31)    >>> f'{user=} {member_since=}'  "user='eric_idle', 7, 31)"

f String format specifiers allow more granular control over the expression results to be displayed :

>>> delta = - member_since >>> f'{user=!s}  {delta.days=:,d}'  'user=eric_idle  delta.days=16,075'

= The specifier will output the entire expression , So as to demonstrate the calculation process in detail :

>>> print(f'{theta=}  {cos(radians(theta))=:.3f}')  theta=30  cos(radians(theta))=0.866
4.typing Module improvements

Python Is a dynamically typed language , But it can go through typing Module add type prompt , So that third-party tools can verify Python Code .Python 3.8 to typing Added some new elements , So it can support more robust checking :

  • final Decorator and Final The type designation indicates , Objects that are decorated or annotated should not be overwritten at any time 、 Inherit , Nor can it be reassigned .
  • Literal Type restricts an expression to a specific value or list of values ( Not necessarily values of the same type ).
  • TypedDict Can be used to create dictionaries , The value of a particular key is restricted to one or more types . Note that these restrictions are only used to determine the validity of values at compile time , It cannot be restricted at runtime .
5. Multi process shared memory

multiprocessing Module addition SharedMemory class , Can be in different Python Create shared memory areas between cities .

In the old version of Python in , Data can only be shared between processes by writing to a file 、 Send via network socket , Or adopt Python Of pickle Module serialization, etc . Shared memory provides a faster way to pass data between processes , Thus making Python More efficient multiprocessor and multi-core programming .

Shared memory fragments can be allocated as simple byte areas , It can also be assigned as a list like object that cannot be modified , It can save the number type 、 character string 、 Byte object 、None Objects, etc Python object .

6. The new version of the pickle agreement

Python Of pickle Modules provide a way to serialize and deserialize Python Methods of data structures or instances , You can save the dictionary as it is for later reading . Different versions Python Supported by pickle Different agreements , and 3.8 Version has a wider range of support 、 More powerful 、 More efficient serialization .

Python 3.8 Introduced 5 edition pickle The protocol can be used in a new way pickle object , It can support Python Buffer protocol , Such as bytes、memoryviews or Numpy array etc. . new pickle Avoid a lot in pickle These objects are used for memory copy operations .

NumPy、Apache Arrow Wait for external libraries in their respective Python The binding supports new pickle agreement . new pickle It can also be used as Python 3.6 and 3.7 The plug-in uses , It can be downloaded from PyPI Installation on .

7. Performance improvements
  • Many built-in methods and functions are faster 20%~50%, Because many previous functions need unnecessary parameter conversion .
  • A new opcode Caching can speed up specific instructions in the interpreter . however , At present, only LOAD_GLOBAL opcode, Its speed is improved 40%. Similar optimizations will be implemented in future versions .
  • File copying operations are as follows: shutil.copyfile() and shutil.copytree() Now use platform specific calls and other optimizations , To improve the operation speed .
  • Newly created lists are now on average smaller than before 12%, This is due to the fact that the list constructor knows the length of the list in advance , It can be optimized .
  • Python 3.8 Middle new class ( Such as class A(object)) The write operation in the class variable of becomes faster .operator.itemgetter() and collections.namedtuple() Also get speed optimization .

Four .Python 3.9 New characteristics

1. Dictionary update and merge

Dictionary adds two new operators :「|」 and 「|=」.「|」 Operator is used to merge dictionaries ,「|=」 The operator is used to update the dictionary .

Dictionary merge :

>>> a = {‘farhad’: 1, 'blog’: 2,'python’: 3} >>> b = {’farhad’: 'malik’,'topic’: 'python3.9’}>>> a | b {’blog’: 2, 'python’: 3, ’farhad’:’malik’,'topic’: 'python3.9’} >>> b | a {’farhad’: 1,’blog’: 2, 'python’: 3,'topic’:’python3.9’ }

Dictionary update :

>>> a |= b >>> a {’blog’: 2, 'python’: 3,’farhad’:’malik’}
2. be based on PEG High performance parser for

Python 3.9 It is proposed to use high-performance and stable PEG Instead of the current parser based on LL(1) Of Python Parser .

Current CPython The parser is based on LL(1),LL(1) The parser is a top-down parser , It parses the input from left to right .

Python 3.9 The proposal will LL(1) Replace with new based on PEG The parser , This means that it will release the current LL(1) Grammar is right Python The limitation of . Besides , The current parser fixes many of the hack. therefore , In the long run , This reduces maintenance costs .

3. New string functions : Remove prefixes and suffixes

Python 3.9 Add two new functions to str object :

The first function removes the prefix :str.removeprefix(prefix)

The second function removes the suffix :str.removesuffix(suffix)

'farhad_python'.removeprefix('farhad') #returns python 'farhad_python'.removesuffix('python') #returns farhad
4. Provide type hints for built-in generic types

Python 3.9 By deleting the parallel type hierarchy , Make annotating programs easier .Python 3.9 Support typing Generic syntax in all standard collections of modules .

We can list or dict Directly as type annotations for lists and dictionaries , Instead of relying on typing.List perhaps typing.Dict. therefore , The code now looks simpler , And easier to understand and explain .

def print_value(input: str): print(input) # We would get notified if the input is not a string
5.DateTime Support IANA The time zone

zoneinfo Modules are created to support IANA Time Zone Database . Yes IANA Time zone database support has been added to the standard library .

IANA The time zone is often called tz or zone info. There are many with different search paths IANA The time zone , Used to date-time Object designation IANA The time zone . for example , We can put the right datetime Object to set the search path 「Continent/City」 To set up tzinfo.

dt = datetime(2000, 01, 25, 01, tzinfo=ZoneInfo("Europe/London"))

If you pass in an invalid key , Will trigger zoneinfo.ZoneInfoNotFoundError abnormal .

6.concurrent.futures Optimize

concurrent.futures.Executor.shutdown() A new parameter has been added to cancel_futures. This parameter cancels concurrent tasks that have not been executed . stay Python 3.9 Previously, the main process could only be shut down after all concurrent tasks were completed executor object .

New parameters cancel_futures Has been added to ThreadPoolExecutor as well as ProcessPoolExecutor. The way it works is : When the value of the parameter is True when , Calling shutdown() Function to cancel all pending tasks .

7. Asynchronous programming and multiprocess optimization

Python 3.9 For asynchronous programming (asyncio) And multi process library .

1) Because of security considerations ,asyncio.loop.createdatagramendpoint() Parameters are no longer supported reuse_address.

2) Added coroutines、shutdowndefaultexecutor() and asyncio.tothread() .shutdowndefaultexecutor Responsible for turning off the default executor,asyncio.tothread() Mainly used to run in a single thread IO Intensive function , To avoid a cycle of events .

On the improvement of multi process library ,Python 3.9 towards multiprocessing.SimpleQueue Class adds new methods close().

This method can explicitly close the queue . This will ensure that the queue is closed and does not stay longer than expected . It is worth noting that , Once the queue is closed , Can't call get()、put() and empty() Method .

8. Unified package import error

Python3.9 Previous versions are imported Python The main problem with the library is : When the relative Import exceeds its top-level package ,Python Inconsistent import behavior in .

builtins.import() trigger ValueError, and importlib.import() trigger ImportError.Python3.9 Version is fixed , It will lead to ImportError.

9. Generate random bytes

Python 3.9 New functions have been added in version random.Random.randbytes(). This function can be used to generate random bytes .

Python Support the generation of random numbers , But what if you need to generate random bytes ? stay Python3.9 Before the release , Developers must find ways to generate random bytes . Although they can use os.getrandom()、os.urandom() perhaps secrets.token_bytes() To generate random bytes , But these methods cannot generate pseudo-random patterns .

To ensure that random numbers are generated in accordance with the expected behavior , And the process can be repeated , Developers often put seeds (seed) And random.Random Use modules together . therefore ,Python 3.9 Added random.Random.randbytes() Method , Generate random bytes in a controlled manner .

10. Fix string replacement function

stay Python 3.9 Before the release , For all nonzero n,"".replace("",s,n) Returns an empty string instead of a s. This error confuses the user , And lead to inconsistent application behavior .

Python 3.9 Fixed the problem , No matter n Is it 0, The results are consistent with "".replace("", s) Agreement .

"".replace("", "blog", 1) Returns ’'One would expect to see blog"".replace("","|", 1) Returns ’'One would expect to see |"".replace("","prefix") Howver returns ’prefix'

5、 ... and . More new features

If you want to know more details , Please refer to the official documents .

copyright notice
author[Good pie notes],Please bring the original link to reprint, thank you.

Random recommended