current position:Home>Python modular package management and import mechanism
Python modular package management and import mechanism
2022-01-31 15:18:06 【Mnio steamed stuffed bun】
We are learning python When , Most of them are from print("hello,world")
Start , This line of code , Knocking on the door of every engineer's new world !
Then we began to learn grammar 、 Variable 、 function 、 Under controlled conditions 、 data structure 、 object-oriented , Then I can't wait to talk to bug dealing . In the process , A friend has been silently accompanying us , But we never pay attention to it . He is ours import
brother
import os
Copy code
In our code , It plays an indispensable role . But few people really understand it . Because of him ,python The power of can be brought into play , Today, let's take a good look at it .
1. Modular programming
In our real project , The amount of code may reach hundreds of thousands 、 Millions of lines . If we put these hundreds of thousands 、 Millions of lines of code are written in a file . The consequences are very serious , First, our code file will be very large , The second is to find our code through the naked eye , It's also very difficult , Let alone understand the code logic .
The idea of modular programming has emerged . Modular programming helps developers make overall planning and division of labor , And improve code flexibility and maintainability . For example, many engineers work together to develop the same system .A Logic for login function ,B Do the logic of the registration function ,C The logic of user management . These codes are separate , Inherit it into the same system through modular assembly .
Modular programming : The code of a complete system , Split into small modules
for instance :
(1) Non modular projects : There is only one project py file main.py
def login():
# Here, complete the code editing of login logic
pass
def register():
# Here, complete the coding of registration logic
pass
def user_manage():
# Here we complete the coding of user management
pass
if __main__ == "__main__":
login()
register()
user_manage()
Copy code
(2) Modular projects : Included in the project main.py
、login.py
、register.py
、user_manage.py
-
login.py
def login(): # Here, complete the code editing of login logic pass Copy code
-
register.py
def register(): # Here, complete the coding of registration logic pass Copy code
-
user_manage.py
def user_manage(): # Here we complete the coding of user management pass Copy code
-
main.py
from login import login from register import register from user_manage import user_manage if __main__ == "__main__": login() register() user_manage() Copy code
Actually ,Python So powerful , This feature plays a big role . We don't need to write many functions ourselves , By importing modules written by others , We can use it directly , This can greatly provide efficiency . For example, we want to write a reptile , Suppose there is no modular help , We need to go from 0 Start writing ,http request 、TCP Connect 、 Return processing 、 Underlying packet encapsulation 、 analysis . But with modularity , We only need one line of code , Open the box , Is it very convenient
import requests
Copy code
2. Python Module in
Modular programming , So in Python in , What exactly does our module mean ? stay Python in , We need to distinguish between several concepts :
- modular : One suffix is
.py
The code file is a module - package : One contains a lot of
.py
File folder .(Python3.3 Previously, this folder must contain__init__.py
file ) - library : It may consist of multiple packages and modules , It can be considered as the packaging of a complete project .
3. import sentence
3.1 Import from module
(1) Full import :
Full import will import all global variables in the module 、 function 、 Classes and so on are all imported .
There are two ways to import all the contents of a module :
import xxx
import test
print(dir(test)
# Check what has been imported
# Find what we define Hello class 、hello function 、name All variables are imported
# And there are other things
['Hello', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'hello', 'name']
test.hello()
print(test.name)
# The above method needs to use : Module name . Variable name ( Method name ) quote , Because it just introduces the whole module , The contents of the module are not introduced separately
Copy code
from xxx import *
from test import *
hello()
Hello()
print(name)
# The above can be used directly , because from test import * Have already put test All contents of the module are imported separately .
Copy code
(2) Local import
from xxx import xxx
from test import hello # Only from test Import in module hello function , Nothing else
from test import hello, Hello # Import multiple
hello()
Copy code
Set an alias for the imported content :from xxx import xxx as yyy
from test import hello as hello_func # from test Import in module hello function , And set the alias to :hello_func
hello_func()
Copy code
3.2 from python Import
|——test_package
|——__init__.py
|——test.py
|——test2.py
|——main.py
Copy code
main.py:
Let's guess , Can the following two pieces of code run ?
import test_package
test_package.test.hello()
Traceback (most recent call last):
File "/app/util-python/python-module/main.py", line 8, in <module>
test_package.test.hello()
AttributeError: module 'test_package' has no attribute 'test'
Copy code
from test_package import *
test.hello()
Traceback (most recent call last):
File "/app/util-python/python-module/main.py", line 10, in <module>
test.hello()
AttributeError: module 'test' has no attribute 'hello'
Copy code
See here , Maybe there are three question marks on everyone's face ??? Why is it difficult to put the method of importing modules in the import package ?
In fact! , our python When importing a module , Will take our module .py
Execute the file once , Then generate a module object , The functions we define in the module 、 Variable 、 Class will be added to the properties of the module object : That's why we can pass test.hello()
, because hello() yes test A property of .
What about importing packages ? We know python Your package is essentially a folder , Folders cannot be compiled and executed . Then why can you import Well ? actually , We are import When it comes to a bag , The execution is the... In this package __init__.py
file , It can also be understood as importing packages , Just imported __init__.py
Because we __init__.py
It's empty. , So we introduced a lonely .
(1) By importing modules :
from test_package import test
test.hello()
from test_package.test import hello
hello()
Copy code
(2) By adding __all__
attribute :
__init__.py
file : add to __all__
attribute
__all__ = ["test", "test2"]
Copy code
main.py file :
from test_package import *
# By means of __init__.py It defines __all__ attribute , When importing , You can import all the modules listed in this attribute
test.hello()
test2.hello2()
Copy code
__all__
It is a convention to expose interfaces for modules , To provide ” White list “ Expose the interface in the form of . If you define__all__
, Used in other documentsfrom xxx import *
When importing the file , Only import__all__
Members listed
__all__
For use onlyfrom module import *
This applies .
4. Dynamic import
Above, we introduced the more mainstream import
Statement import , Actually in python There are other import methods in
4.1 __import__()
__import__()
Function can be used to import modules . In fact, when we use import Import Python Module time , The default call is __import__()
function . It is rare to use this function directly , It is generally used to dynamically load modules .
__import__(name, globals=None, locals=None, fromlist=(), level=0)
Parameters :
- name: Module name to import , Available variables
- globals and locals: Default values are usually used . Using the given globals and locals Variable to determine how to resolve in a package context name
- fromlist: Specify the name of the sub module or object to import , They are imported from the module by name
- leve: Specify how to import the module .level by 0 Absolutely import ; level A positive value indicates relative to the call __import __() Module directory , Number of parent directories to search
os_obj = __import__("os")
print(os_obj.getcwd())
Copy code
os_obj = __import__("test_package.test")
os_obj.test.hello()
Copy code
4.2 importlib
importlib yes Python A standard library in ,importlib It can provide a very comprehensive function
import importlib
myos=importlib.import_module("os")
myos.getcwd()
Copy code
4.3 A usage scenario
A timed task scenario , There are many such scheduled tasks stored in the database :cmdb.tasks.get_host_info, It means calling cmdb Under bag tasks Under the module of get_host_info function . How do we achieve , Through a string in this format , To call the corresponding function ?
You can think about how to do .
def exec_task(task_name):
if not task_name:
return -1, "task name must not None"
try:
module_name = task_name.rsplit(".", 1)[0]
method_name = task_name.rsplit(".", 1)[1]
# Dynamic import tasks modular
module_obj = __import__(module_name)
if not hasattr(module_obj, method_name):
return -1, "function not found"
except:
return -1, "has Error"
task = getattr(module_obj, method_name)
task()
Copy code
5. Search path
I wonder if you have ever thought about such a problem , When we import a module or a package ,python Where did you find this module ?
Python The path of the search module is composed of four parts :
- The main directory of the program 、
- PATHONPATH Catalog
- Standard link library Directory
- .pth File directory ,
The paths of these four parts are stored in sys.path In the list .
pth file :pth Files are used to add additional sys.path namely python Search path , Generally in github The package downloaded on will have a setup.py, The execution of this file will occur in ( At present python In the environment site-packages Folder generation ) One .pth file
import sys
print(sys.path)
[
'/app/util-python/python-module', # The directory where the current program is located
'/usr/lib/python37.zip',
'/usr/lib/python3.7',
'/usr/lib/python3.7/lib-dynload',
'/app/python-virtualenv/aioms-env/lib/python3.7/site-packages' # Download the third-party package directory
]
Copy code
When our When importing a package ,python The interpreter searches these directories in turn . If not found in these directories , The program will report an error
Suppose there's a file : /app/test_pack.py
Our current program path is :/app/util-python/python-module/main.py
import test_pack
Traceback (most recent call last):
File "/app/util-python/python-module/main.py", line 5, in <module>
import test_pack
ModuleNotFoundError: No module named 'test_pack'
Copy code
There was no accident , Wrong report , I can't find this bag
import sys
sys.path.append('/app')
import test_pack
test_pack.hello()
# Test import custom path
print(sys.path)
[
'/app/util-python/python-module',
'/usr/lib/python37.zip',
'/usr/lib/python3.7',
'/usr/lib/python3.7/lib-dynload',
'/app/python-virtualenv/aioms-env/lib/python3.7/site-packages',
'/app' # We found an additional search directory , It found our... In this directory test_pack.py
]
Copy code
therefore : The search paths of modules and packages are not fixed , We can customize it , Of course, the above method is only temporary
therefore , When this error occurs in our program :ModuleNotFoundError: No module named 'test_pack'
Problem solving trilogy :
- Check where the path of the downloaded and installed package is
- Use sys.path have a look , Is the downloaded package here
6. Relative Import and absolute import
Suppose the package structure :
main.py
packageA/
__init__.py
moduleA.py
moduleB.py
packageB/
__init__.py
moduleA.py
moduleB.py
Copy code
about packageA/moduleA.py
:
Absolutely import : All modules import From the “ The root node ” Start . The location of the root node is determined by sys.path The path in determines
from packageA import moduleB
Copy code
Relative Import : Only care about the module location relative to your current directory
- . Current directory peer lookup
- .. Find the parent of the current directory
from . import moduleB
Copy code
7. Cross reference ( Import loop )
What is cross import , The two packages are imported into each other
main.py
package/
__init__.py
moduleA.py
moduleB.py
Copy code
from . import moduleB
Copy code
I don't want to expand too much here , It is recommended that everyone use absolute import to finish ! So as not to dig a hole for yourself
moduleA.py:
from moduleB import hello_b
Copy code
moduleB.py:
from moduleA import hello_a
Copy code
In this : modular A Called module B A method of 、 And modules B Also called the module A A method of , This is called cross import
First, let's see what problems this will lead to
ImportError: cannot import name 'hello_b' from 'moduleB'
Copy code
This will throw an exception . The reason for the abnormality is that : When importing ,moduleA Need to access moduleB Of hello_b, But no hello_b It's not initialized yet . So an exception will be thrown
Generally speaking : This is due to large Python In Engineering , Improper architecture design , Mutual reference between modules
terms of settlement :
- Structure optimization , Dereference
- import The statement is placed at the end of the module
- import Statement placed in a function
8. Install third party package
What is a third-party package ? There is a popular saying in the programming circle :“ Don't make wheels over and over again !”. The wheel here is actually our third-party package . When we want to make a car , We can just use the existing parts to assemble them . There's no need to start from 0 Start , To build the wheels 、 Engine, etc . Because others have made it .
In the computer industry , Someone else built the wheel , There must always be a place to save , It can be seen and used by other users . This requires an authoritative tripartite organization to manage these wheels .
PyPI It's such a role , PyPI(Python Package Index) yes python The official third-party warehouse , Everyone can download third-party libraries or upload their own developed libraries to PyPI.
How to manage these three-party packages ? There are many methods on the market . But the most widely used is pip
pip It's a modern , General purpose Python Package management tools , The tool provides the right Python Bag search 、 download 、 install 、 Uninstalled features
7.1 pip install
# see pip edition , Can be judged pip Whether to install
pip -v
Copy code
In general , We downloaded it from the official website python After the installation , I will bring this tool with me , We don't have to worry too much , however , If it's too bad , Yours Python There happens to be no pip This tool , What shall I do? ?
about linux user :
$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py # Download the installation script
$ sudo python get-pip.py # Run the installation script
Copy code
sudo apt-get install python-pip
Copy code
window Users are similar to , Just go to the official website to download pip Installation package , Reuse python Can be installed
7.2 Install third party package
pip install Django # Install the latest version
pip install Django==1.0.4 # Specify the version
pip install Django>=1.0.1 # Specify the minimum version
pip install Django<=1.0.1 # Specify the maximum version
Copy code
because PyPI The image source is foreign , Sometimes the download is very slow , At this time, we can use the domestic image source
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple Django
Copy code
Domestic image source address :
- tsinghua :pypi.tuna.tsinghua.edu.cn/simple
- Alibaba cloud :mirrors.aliyun.com/pypi/simple…
- douban :pypi.douban.com/simple/
- ...... There are others to search for
7.3 Other operating
pip uninstall Django # Uninstall installed Libraries
pip install --upgrade Djano==2.0.1 # Upgrade installed packages
pip list # List installed libraries
pip freeze > requirements.txt # Add the current project dependency to the exported text file
pip install -r requirements.txt # Install according to the dependencies in the text file exported above
Copy code
Pay attention to the official account of Xiaobian : Learn secretly , Kill them
copyright notice
author[Mnio steamed stuffed bun],Please bring the original link to reprint, thank you.
https://en.pythonmana.com/2022/01/202201311518042732.html
The sidebar is recommended
- Django (make an epidemic data report)
- Daily python, Part 8 - if statement
- Django model class 1
- The same Python code draws many different cherry trees. Which one do you like?
- Python code reading (Chapter 54): Fibonacci sequence
- Django model class 2
- Python crawler Basics
- Mapping 3D model surface distances using Python VTK
- How to implement encrypted message signature and verification in Python -- HMAC
- leetcode 1945. Sum of Digits of String After Convert(python)
guess what you like
-
leetcode 2062. Count Vowel Substrings of a String(python)
-
Analysis of Matplotlib module of Python visualization
-
Django permission management
-
Python integrated programming -- visual hot search list and new epidemic situation map
-
[Python data collection] scripy realizes picture download
-
Python interface automation test framework (basic part) -- loop statement of process control for & while
-
Daily python, Chapter 9, while loop
-
Van * Python | save the crawled data with docx and PDF
-
Five life saving Python tips
-
Django frequency control
Random recommended
- Python - convert Matplotlib image to numpy Array or PIL Image
- Python and Java crawl personal blog information and export it to excel
- Using class decorators in Python
- Untested Python code is not far from crashing
- Python efficient derivation (8)
- Python requests Library
- leetcode 2047. Number of Valid Words in a Sentence(python)
- leetcode 2027. Minimum Moves to Convert String(python)
- How IOS developers learn Python Programming 5 - data types 2
- leetcode 1971. Find if Path Exists in Graph(python)
- leetcode 1984. Minimum Difference Between Highest and Lowest of K Scores(python)
- Python interface automation test framework (basic) -- basic syntax
- Detailed explanation of Python derivation
- Python reptile lesson 2-9 Chinese monster database. It is found that there is a classification of color (he) desire (Xie) monsters during operation
- A brief note on the method of creating Python virtual environment in Intranet Environment
- [worth collecting] for Python beginners, sort out the common errors of beginners + Python Mini applet! (code attached)
- [Python souvenir book] two people in one room have three meals and four seasons: 'how many years is it only XX years away from a hundred years of good marriage' ~?? Just come in and have a look.
- The unknown side of Python functions
- Python based interface automation test project, complete actual project, with source code sharing
- A python artifact handles automatic chart color matching
- Python crawls the map of Gaode and the weather conditions of each city
- leetcode 1275. Find Winner on a Tic Tac Toe Game(python)
- leetcode 2016. Maximum Difference Between Increasing Elements(python)
- Run through Python date and time processing (Part 2)
- Application of urllib package in Python
- Django API Version (II)
- Python utility module playsound
- Database addition, deletion, modification and query of Python Sqlalchemy basic operation
- Tiobe November programming language ranking: Python surpasses C language to become the first! PHP is about to fall out of the top ten?
- Learn how to use opencv and python to realize face recognition!
- Using OpenCV and python to identify credit card numbers
- Principle of Python Apriori algorithm (11)
- Python AI steals your voice in 5 seconds
- A glance at Python's file processing (Part 1)
- Python cloud cat
- Python crawler actual combat, pyecharts module, python data analysis tells you which goods are popular on free fish~
- Using pandas to implement SQL group_ concat
- How IOS developers learn Python Programming 8 - set type 3
- windows10+apache2. 4 + Django deployment
- Django parser