current position:Home>Python crawling material commune picture

Python crawling material commune picture

2022-05-15 02:35:33Emilyzhai


import requests
import re
# import urllib
# from urllib import request,error

url = "https://www.tooopen.com/aicdr/1096_1100_1_1.aspx"
def askURL(url):
    header = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36",
        "Referer": "https://www.tooopen.com/aicdr/1096_1100_1_2.aspx",
        "Cookie": "ASP.NET_SessionId=4dff2ec6-6f07-47cc-8de1-8f3478edf6e8; Hm_lvt_d3ac2f8840ead98242d6205eeff29cb4=1652173245; Hm_lpvt_d3ac2f8840ead98242d6205eeff29cb4=1652173337; RefreshFilter=http://www.tooopen.com/ajax/gethistory?callback=jQuery183003647995761540557_1652173336490&_=1652173336598"
    }
    res = requests.get(url,verify=False,headers=header)
    page_content = res.text
    obj = re.compile(r'<a class="pic".*?<img src="(?P<img>.*?)"',re.S)
    result = obj.finditer(page_content)
    m = 0
    imgs = []
    for it in result:
        imgs.append(it.group('img'))

    ### Use urllib How to download pictures 
    # m=0
    # for img in imgs:
    #     urllib.request.urlretrieve(img, "d:/download2/" + str(m) + '.jpg')  #  Download pictures to local 
    
    ## Use requests How to download pictures 

    for imgurl in imgs:
        imgres = requests.get(imgurl, verify=False, headers=header)
        with open("d:/download2/" + str(m) + '.jpg','wb') as f:
            f.write(imgres.content)
        m += 1

askURL(url)

The code provides two ways to download pictures , Download pictures , encounter 403 error , Add the following code to solve , Be careful Referer and Cookie Add together .

 header = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36",
        "Referer": "https://www.tooopen.com/aicdr/1096_1100_1_2.aspx",
        "Cookie": "ASP.NET_SessionId=4dff2ec6-6f07-47cc-8de1-8f3478edf6e8; Hm_lvt_d3ac2f8840ead98242d6205eeff29cb4=1652173245; Hm_lpvt_d3ac2f8840ead98242d6205eeff29cb4=1652173337; RefreshFilter=http://www.tooopen.com/ajax/gethistory?callback=jQuery183003647995761540557_1652173336490&_=1652173336598"
    }

copyright notice
author[Emilyzhai],Please bring the original link to reprint, thank you.
https://en.pythonmana.com/2022/132/202205120518203439.html

Random recommended