「 This is my participation 11 The fourth of the yuegengwen challenge 9 God , Check out the activity details ：2021 One last more challenge 」

preview

One 、 The experiment purpose

Through this experiment Python The basic knowledge and third-party database in the language have been comprehensively applied . Complete the visual hot search list and the new map of domestic epidemic situation , Improve students' programming ability and analyze problems 、 Ability to solve problems .

Two 、 Equipment and environment

Hardware ： Multimedia computer Software ：Windows7 or Windows10 operating system 、Python3.X Software .

3、 ... and 、 Experimental content

1、 Experimental content ① Use python web frame flask build web project ② Use crawler technology to complete information acquisition ③ Use python The basic knowledge base completes data conversion and data analysis ④ Use jieba The library does word frequency analysis on hot search ⑤ Use jQuery The framework and HTML、css、JavaScript and echarts Complete the front-end page design

2、 The final result is output . requirement ： The output format should be intuitive 、 Clear and generous 、 Format specification .

Four 、 Experimental results and Analysis

1、 Experimental operation process and analysis

# Build home page routing path , And load index.html Page and transfer data 
@app.route('/Hot_Bot')
def Hot_Bot():
    data=hotBot()
    return render_template('index.html',form=data,title=data.title)

# Build word frequency pages and routing paths , And load test.html Page and transfer data 
@app.route('/cipin')
def cipin():
    data=spider.sum_hot_word()
    print(data)
    return render_template("test.html",form=data)
 Copy code 

 Take Weibo and Zhihu as examples ：

def weibo():
    hot=[]
    name=[]
    value=[]
    url='https://weibo.com/ajax/statuses/hot_band'
    header={
        'cookie': 'UOR=mp.weixin.qq.com,s.weibo.com,mp.weixin.qq.com; SINAGLOBAL=753710676249.8569.1621750150925; SUB=_2AkMXpEktf8NxqwJRmP4Tz2zkZYh3wwHEieKh-Lj2JRMxHRl-yT9jqhAztRB6PCRnwgM0JsVYPTwi5DuGI3N0YpgPChkI; SUBP=0033WrSXqPxfM72-Ws9jqgMF55529P9D9WhpfXwV9S99niOF07XLn8y7; WBPSESS=kErNolfXeoisUDB3d9TFH-1YhWD5pAkKF4olmR2WdEz_79spnMzQbf2Kt92964Tdvd3fcKY1c8a_Sd6CbCiw6P0wyFuEu1GQri6NrQ6_oBLuAYd8HR3zZI8_M6QfSsHD; ULV=1635245354703:3:1:1:6287771993091.978.1635245354698:1626916415441; XSRF-TOKEN=_LdujowesXEM4itQidVLNlJj',
        'accept - encoding': 'gzip, deflate, br',
        'user - agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36',
        'referer': 'https://www.baidu.com/link?url=oBKJ9ZCKdgcrDL-WKTnXNgHhk2kNw6JfV0tShTgwv3KYkUracwd2FG6kuIrShm5b2aJDHZKZVgYG8QgZWSM-Ha&wd=&eqid=f74c877700049c31000000066177abe3'
    }
    req = requests.get(url,headers=header).text
    soup = BeautifulSoup(req, "lxml")
    #hot_word = re.findall('word.*?,',req)
    hot_word=json.loads(req)['data']['band_list']
        for i in range(len(hot_word)):
        hot.append({"name":hot_word[i]['word'][0:10],"value":hot_word[i]["num"]})
        name.append(hot_word[i]['word'])
        value.append(hot_word[i]["num"])
    return hot[0:3],name,value
 Copy code 

def zhihu():
    hot = []
        browser=webdriver.Chrome('chromedriver.exe')
    browser.get('https://www.zhihu.com/topsearch')
    browser.refresh()
    elements=browser.find_elements_by_class_name('TopSearchMain-title')
    for i in elements:
        hot.append(i.text)
    return hot
 Copy code 

 Take word frequency analysis as an example to show some front-end page codes ：
<div id="main" style="width: 600px;height: 800px;"></div>
<script> var ectest = echarts.init(document.getElementById("main")); var ec_right2_option = { // backgroundColor: '#515151', title: { text: " Today's epidemic situation is hot ", textStyle: { color: 'white', }, left: 'left' }, tooltip: { show: false }, series: [{ type: 'wordCloud', // drawOutOfBound:true, gridSize: 1, sizeRange: [12, 55], rotationRange: [-45, 0, 45, 90], // maskImage: maskImage, textStyle: { normal: { color: function () { return 'rgb(' + Math.round(Math.random() * 255) + ', ' + Math.round(Math.random() * 255) + ', ' + Math.round(Math.random() * 255) + ')' } } }, right: null, bottom: null, data: ddd }] } ectest.setOption(ec_right2_option); </script>
 Copy code 

2、 Running results

The main page is shown below ： In the middle is the newly added map of domestic epidemic situation, and the number of people is dynamically displayed , The top left is Baidu hot search list top3、 The bottom left is the microblog hot search list top3、 The middle and upper are the weather conditions of the day 、 The top right is the comparison between microblog and Baidu hot search . At the bottom right is the hot search address and word frequency statistics of the three platforms .

The following figure shows Baidu 、 Microblogging 、 Know the hot search word frequency statistics of the three platforms .

3、 Experience

Through this course design, I reviewed again python Some basic knowledge of and a further understanding of front-end technology have made me more clear about my future direction . And also realize the importance of the overall architecture when building the project , The most important thing is to have a deeper understanding of python The advantages and disadvantages of technology for building websites python The practicality of language . It has played a good guiding role in the future development . At the same time, I realized my weaknesses , For example, when designing front-end pages, you should jQuery and JavaScript The application of technology is very unskilled and right echarts The chart selection is also relatively simple , When selecting the city coordinate mark of epidemic disease, it is necessary to symbolSize The setting of is not well controlled, which directly leads to the failure of the final product to achieve the expected effect , And right python The mastery of basic knowledge is not particularly solid for jieba The use of the library needs to be improved . I'll fill in the weak points later , Strive to be a full stack Technician .