current position:Home>Python crawler - ETF fund acquisition
Python crawler - ETF fund acquisition
2022-02-01 03:36:44 【first quarter of the moon】
This is my participation 11 The fourth of the yuegengwen challenge 5 God , Check out the activity details :2021 One last more challenge
Light is always short , Darkness is eternity , You pompous light , How can we understand the depth of darkness ?
1 Preface
The change information of the fund has been introduced before , But these funds are off-site , Today we are going to introduce an investment product with threshold -ETF. Only players who open securities accounts can enter the game ,ETF It is an exchange traded fund , You can trade in intraday , Trading is a little stronger than OTC funds , Then talk less , Let's get down to business right now .
2 ETF List and abbreviations
ETF The acquisition method of fund changes and basic information is the same as that of OTC funds , How to obtain more comprehensive ETF What about the fund list ?
# Get a list of fund information
http://fund.eastmoney.com/data/fbsfundranking.html
Copy code
Here are ETF The information displayed in the information list .
ETF When trading on the floor , There is usually an abbreviation , The way to get the abbreviation is a little more troublesome , Need to visit a page , And then through bs4 To parse elements to get .
# Through the analysis of , We can find that the fund code prefix represents its market ,5 Shanghai market 1- Shenzhen market , To real estate ETF And photovoltaic ETF For example
http://quote.eastmoney.com/sz159707.html
http://quote.eastmoney.com/sh515790.html
Copy code
3 ETF information acquisition
3.1 ETF Get a list of letters
ETF List information by accessing the list, we find that when accessing the list data , It's a request api Interface to the background , Then it returns a response message to the front end .
http://fund.eastmoney.com/data/rankhandler.aspx?op=ph&dt=fb&ft=ct&rs=&gs=0&sc=zzf&st=desc&pi=1&pn=50
Copy code
I feel very happy to see here , Don't parse html The file , When passed request Use get When getting data by , It is found that no access permission is returned , I think maybe I didn't carry cookie Why , But I didn't log in , Maybe the request header needs to carry some page information , therefore , After trying , Finally, it is determined that the information to be carried is :
headers = {
'Host': 'fund.eastmoney.com',
'Referer': 'http://fund.eastmoney.com/data/fbsfundranking.html'
}
Copy code
Finally, the code for obtaining the fund list should be written like this :
The results obtained after debugging are shown in the figure below :
3.2 Get the abbreviation of the Fund
Getting the abbreviation of the fund is relatively simple , Through analysis, we found that , The abbreviation is located in <span class="quote_title_0 wryh"> Photovoltaic ETF</span>
in , By visiting the page to get the element, you can get the description of the abbreviation . The specific code is shown in the figure below :
4 The end result shows
After obtaining the list of funds and obtaining the abbreviation of funds , We got the final result, as shown in the figure below , The purpose of obtaining information has been achieved :
In the future, we will share the fund information with ETF The information is merged and stored in the database , Facilitate subsequent data analysis .
copyright notice
author[first quarter of the moon],Please bring the original link to reprint, thank you.
https://en.pythonmana.com/2022/02/202202010336415016.html
The sidebar is recommended
- Python avatar animation, come and generate your own animation avatar
- leetcode 1884. Egg Drop With 2 Eggs and N Floors(python)
- leetcode 1910. Remove All Occurrences of a Substring(python)
- Python and binary
- First acquaintance with Python class
- [Python data collection] scrapy book acquisition and coding analysis
- Python crawler from introduction to mastery (IV) extracting information from web pages
- Python crawler from entry to mastery (III) implementation of simple crawler
- The apscheduler module in Python implements scheduled tasks
- 1379. Find the same node in the cloned binary tree (Java / C + + / Python)
guess what you like
-
Python connects redis, singleton and thread pool, and resolves problems encountered
-
Python from 0 to 1 (day 11) - Python data application 1
-
Python bisect module
-
Python + OpenGL realizes real-time interactive writing on blocks with B-spline curves
-
Use the properties of Python VTK implicit functions to select and cut data
-
Learn these 10000 passages and become a humorous person in the IT workplace. Python crawler lessons 8-9
-
leetcode 986. Interval List Intersections(python)
-
leetcode 1860. Incremental Memory Leak(python)
-
How to teach yourself Python? How long will it take?
-
Python Matplotlib drawing pie chart
Random recommended
- Django paging (II)
- Concurrent. For Python concurrent programming Futures or multiprocessing?
- Programmers over the age of 25 can't know a few Chinese herbal medicines. Python crawler lessons 9-9
- Python crawler from introduction to pit full series of tutorials (detailed tutorial + various practical combat)
- The second bullet of class in Python
- Python object oriented programming 03: class inheritance and its derived terms
- How IOS developers learn Python Programming 13 - function 4
- Python crawler from introduction to mastery (VI) form and crawler login
- Python crawler from entry to mastery (V) challenges of dynamic web pages
- Deeply understand pandas to read excel, TXT, CSV files and other commands
- Daily python, Chapter 18, class
- "I just want to collect some plain photos in Python for machine learning," he said. "I believe you a ghost!"
- Django view
- Python implements filtering emoticons in text
- When winter comes, python chooses a coat with temperament for mom! Otherwise, there's really no way to start!
- Python crawler - get fund change information
- Highlight actor using Python VTK
- Python crawler actual combat: crawling southern weekend news articles
- leetcode 406. Queue Reconstruction by Height(python)
- leetcode 1043. Partition Array for Maximum Sum (python)
- Python * * packaging and unpacking details
- Python realizes weather query function
- Python from 0 to 1 (day 12) - Python data application 2 (STR function)
- Python from 0 to 1 (day 13) - Python data application 3
- Numpy common operations of Python data analysis series Chapter 8
- How to implement mockserver [Python version]
- Van * Python! Write an article and publish the script on multiple platforms
- Python data analysis - file reading
- Python data De duplication and missing value processing
- Python office automation - play with browser
- Python series tutorial 127 -- Reference vs copy
- Control flow in Python: break and continue
- Teach you how to extract tables in PDF with Python
- leetcode 889. Construct Binary Tree from Preorder and Postorder Traversal(python)
- leetcode 1338. Reduce Array Size to The Half(python)
- Object oriented and exception handling in Python
- How to configure load balancing for Django service
- How to embed Python in go
- Python Matplotlib drawing graphics
- Python object-oriented programming 05: concluding summary of classes and objects