current position:Home>Asynchronous programming in Python

Asynchronous programming in Python

2022-02-02 10:48:54 Listen to the sea without wind

One 、 Asynchronous programming overview

Asynchronous programming is a mode of concurrent programming , Its focus is to schedule the execution and waiting time between different tasks , Reduce the execution time of the whole program by reducing the idle time of the processor ; The biggest difference between asynchronous programming and synchronous programming model is its task switching , When you encounter a task that needs to wait for a long time , We can switch to other tasks ;

Compared with multithreading and multiprocessing programming models , Asynchronous programming is just task scheduling within the same thread , Can't make full use of multi-core CPU The advantages of , So it is especially suitable for IO Blocking tasks ;

python edition 3.9.5

Two 、python Asynchronous framework model

python Provides asyncio Module to support asynchronous programming , It involves coroutines、event loops、futures Three important concepts ;

event loops Mainly responsible for tracking and scheduling all asynchronous tasks , Choreograph specific tasks to be performed at a certain point in time ;

coroutines It is the encapsulation of specific tasks , Is one that can pause during execution and switch to event loops A special type of function that executes a process ; It generally needs to create task Can be event loops Dispatch ;

futures Responsible for bearing coroutines The results of the implementation of , It follows the task in event loops Created by initialization in , And record the execution status of the task with the execution of the task ;

The whole execution process of asynchronous programming framework involves the close cooperation of the three ;

First event loops After starting , The first task to be executed will be obtained from the task queue coroutine, And then create the corresponding task and future;

Then as the task Implementation , When you meet coroutine Where internal tasks need to be switched ,task The execution of will pause and release the execution thread to event loop,event loop Then the next to be executed... Will be obtained coroutine, After relevant initialization , Execute this task;

With event loop Finish executing the last... In the queue coroutine Will switch to the first coroutine;

With task End of execution for ,event loops Will task Clear out the queue , The corresponding execution results will be synchronized to future in , This process will continue until all task end of execution ;

image

3、 ... and 、 Execute multiple overlapping tasks in sequence

Each task will be suspended for a given time , The sequential execution time is the sum of the execution time of each task ;

import time

def count_down(name, delay):
    indents = (ord(name) - ord('A')) * '\t'

    n = 3
    while n:
        time.sleep(delay)
        duration = time.perf_counter() - start
        print('-' * 40)
        print(f'{duration:.4f} \t{indents}{name} = {n}')
        n -= 1


start = time.perf_counter()

count_down('A', 1)
count_down('B', 0.8)
count_down('C', 0.5)
print('-' * 40)
print('Done')

# ----------------------------------------
# 1.0010 	A = 3
# ----------------------------------------
# 2.0019 	A = 2
# ----------------------------------------
# 3.0030 	A = 1
# ----------------------------------------
# 3.8040 		B = 3
# ----------------------------------------
# 4.6050 		B = 2
# ----------------------------------------
# 5.4059 		B = 1
# ----------------------------------------
# 5.9065 			C = 3
# ----------------------------------------
# 6.4072 			C = 2
# ----------------------------------------
# 6.9078 			C = 1
# ----------------------------------------
# Done

Four 、 Asynchronous synchronous code

python Grammatically provides async、await Two keywords to simplify changing synchronous code to asynchronous code ;

async Used in the function def Keyword front , Mark this is a coroutine function ;

await Use in conroutine inside , Used to mark that the execution process needs to be suspended and released event loops;

await The following expression needs to return waitable The object of , for example conroutine、task、future etc. ;

asyncio The module mainly provides operation event loop The way ;

We can go through async take count_down Marked as coroutine, And then use await and asyncio.sleep To achieve asynchronous pause , So as to give control to event loop;

async def count_down(name, delay, start):
    indents = (ord(name) - ord('A')) * '\t'

    n = 3
    while n:
        await asyncio.sleep(delay)
        duration = time.perf_counter() - start
        print('-' * 40)
        print(f'{duration:.4f} \t{indents}{name} = {n}')
        n -= 1

Our definition of asynchrony main Method , Mainly complete task Create and wait for the completion of task execution ;

async def main():
    start = time.perf_counter()
    tasks = [asyncio.create_task(count_down(name,delay,start)) for name, delay in [('A', 1),('B', 0.8),('C', 0.5)]]
    await asyncio.wait(tasks)
    print('-' * 40)
    print('Done')

We can see that the time has become the time to perform the longest task ;

asyncio.run(main())

# ----------------------------------------
# 0.5010 			C = 3
# ----------------------------------------
# 0.8016 		B = 3
# ----------------------------------------
# 1.0011 	A = 3
# ----------------------------------------
# 1.0013 			C = 2
# ----------------------------------------
# 1.5021 			C = 1
# ----------------------------------------
# 1.6026 		B = 2
# ----------------------------------------
# 2.0025 	A = 2
# ----------------------------------------
# 2.4042 		B = 1
# ----------------------------------------
# 3.0038 	A = 1
# ----------------------------------------
# Done

5、 ... and 、 Overcome the specific limitations of asynchronous multithreading

Asynchronous programming requires that the specific task must be coroutine, That is, the method is required to be asynchronous , Otherwise, only the task is completed , To release control to event loop;

python Medium concurent.futures Provides ThreadPoolExecutor and ProcessPoolExecutor, It can be used directly in asynchronous programming , Thus, tasks can be performed in a separate thread or process ;

import time
import asyncio
from concurrent.futures import ThreadPoolExecutor

def count_down(name, delay, start):
    indents = (ord(name) - ord('A')) * '\t'

    n = 3
    while n:
        time.sleep(delay)

        duration = time.perf_counter() - start
        print('-'*40)
        print(f'{duration:.4f} \t{indents}{name} = {n}')
        n -=1

async def main():
    start = time.perf_counter()
    loop = asyncio.get_running_loop()
    executor = ThreadPoolExecutor(max_workers=3)
    fs = [
       loop.run_in_executor(executor, count_down, *args)  for args in [('A', 1, start), ('B', 0.8, start), ('C', 0.5, start)]
    ]

    await asyncio.wait(fs)
    print('-'*40)
    print('Done.')

asyncio.run(main())

# ----------------------------------------
# 0.5087 			C = 3
# ----------------------------------------
# 0.8196 		B = 3
# ----------------------------------------
# 1.0073 	A = 3
# ----------------------------------------
# 1.0234 			C = 2
# ----------------------------------------
# 1.5350 			C = 1
# ----------------------------------------
# 1.6303 		B = 2
# ----------------------------------------
# 2.0193 	A = 2
# ----------------------------------------
# 2.4406 		B = 1
# ----------------------------------------
# 3.0210 	A = 1
# ----------------------------------------
# Done.

copyright notice
author[Listen to the sea without wind],Please bring the original link to reprint, thank you.
https://en.pythonmana.com/2022/02/202202021048535938.html

Random recommended