Python requests Library

grammar

requests.nameofmethod(parameters)
 Copy code 

Background and introduction

_requests_ How many libraries are there GET Methods . The first part and The second part It involves a lot in requests.get() . The point of this article is _GET_ Other methods and error handling .

If it doesn't exist on the computer requests library , Please browse The first part Explanation .

"get " request " The header file "

By default , This value is None . If True , One HTTPS Head message Dictionaries It will be transmitted to the specified URL.

When one HTTP_ request _ When it was launched , One User-Agent The string is transmitted with the request , This string contains the following details of your system .

• Application type
• operating system
• Software supplier
• The software version of the requested user agent

The server uses these details to determine your computer's capabilities .

In this case , This code will send its header information to the server ：

• The first [1] Line import request library
• The first [2] A well formed User-Agent Save the string to _hdrs_ variable
• The first [3] Line trying to connect to _URL_, And set the header information to _hdrs_
• The first [4] Output the header information response to the terminal
• The first [5] Line closes the open connection

Code

import requests
hdrs = {
    "Connection": "keep-alive",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, 
    like Gecko) Chrome/72.0.3626.121 Safari/537.36"}

response = requests.get('https://app.finxter.com', headers=hdrs)
print(response.headers)
response.close()

 Copy code 

Output

{'Server': 'nginx/1.14.0 (Ubuntu)', 'Date': 'Fri, 05 Nov 2021 16:59:19 GMT', 'Content-Type': 'text/html; charset=utf-8',
'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'X-Frame-Options': 'DENY', 'Vary': 'Cookie', 'X-Content-Type-Options': 'nosniff', 'Set-Cookie': 'sessionid=0fb6y6y5d8xoxacstf74ppvacpmt2tin; expires=Fri, 19 Nov 2021 16:59:19 GMT; HttpOnly; Max-Age=19600; Path=/; SameSite=Lax', 'Content-Encoding': 'gzip'}
 Copy code 

Be careful ： This is a great Python function . If you are right about Web Scraping Interested in , You may want to delve further into this topic .

"get " request " Agency "

If you're an avid web Raider , Or you need to keep your privacy online , Using agents is the answer . The agent will your IP The address is hidden from the outside world .

In this case , We get a new... From a free agency service IP Address , And add it to a dictionary .

• The first [1] Line import request library
• The first [2] Set a row URL To _the_url_ Variable
• The first [3] Line adds a new proxy in the form of a dictionary , Up to now
• The first [4] Line attempts to connect to the _URL_, And set the proxy to _my_proxy_
• The first [5] Output the status code response to the terminal
• The first [6] Line close open connection

Code

import requests
the_url  = 'https://somewebsite.com'
my_proxy = {"https": "https:157.245.222.225:3128"}
response = requests.get(the_url, proxies=my_proxy)
print(response.status_code)
response.close()

 Copy code 

Output

200
 Copy code 

"get" request ." flow "

This method is unnecessary . By default , This value is False . If False , The response transmission indicates that the file should be downloaded immediately . If it is True , Then stream the file

• The first [1] Line import request library
• The first [2] On the way _URL_ Set to logo Location , And will stream Set to _True_
• The first [3] The line outputs the status code in response to the terminal
• The first [4] Line close open connection

Code

import requests
response = requests.get('https://app.finxter.com/static/favicon_coffee.png', stream=True)
print(response.status_code)
response.close()
 Copy code 

Output

200
 Copy code 

exception handling

There are a lot of exceptions and request'_s_ Library related

There are two ways to deal with this situation

individualization

In this case , We will Overtime To requests.get() . If the connection or server times out , An exception will occur ：

• The first [1] Line import request library
• The first [2] Line initialized _try_ sentence . The code here will first run
• The first [3] Line trying to connect to URL And set timeout
• The first [4] Output the status code to the terminal
• The first [5] The line closes the open connection
• The first [6] Line is _except_ sentence . In case of _ Overtime _, The code will fall here
• The first [7] The line outputs information to the terminal _Timed Out!_ The script is over

Code

import requests
try:
   response = requests.get('https://app.finxter.com', timeout=(2,4))
   print(response.status_code)
   response.close()
except requests.ConnectTimeout():
    print('Timed Out!')

 Copy code 

Output

200
 Copy code 

All exceptions

_ request _ All exceptions in the library are inherited from requests.exceptions.RequestException. For this example , This code captures all exceptions

• The first [1] Line import request library
• The first [2] Line initialized _try_ sentence . The code here will first run
• The first [3] Line trying to connect to URL And set timeout
• The first [4] Output the status code to the terminal
• The first [5] The line closes the open connection
• The first [6] Line is _except_ sentence , If anything goes wrong , The code will fall here
• The first [7] Line exception information （e） Output to terminal , Script end

Code

import requests
try:
   response = requests.get('https://app.finxter.com', timeout=(2,4))
   print(response.status_code)
   response.close()
except requests.exceptions.RequestException as e:
    print(e)

 Copy code 

Output

200
 Copy code 

in addition

You can also convert the above content into a reusable function , By modifying this code to meet different needs

Code

def error_code(url):
    try:
        response = requests.get('https://app.finxter.c', timeout=(2,4))
    except requests.exceptions.RequestException as e:
        return e
       
nok = error_code('https://app.finxter.com')
print(nok)
 Copy code 

summary

In this article , We learned how to .

• Use header file
• Use an agency
• Use stream
• Implement exception handling