We are learning python When , Most of them are from print("hello,world") Start , This line of code , Knocking on the door of every engineer's new world ！

Then we began to learn grammar 、 Variable 、 function 、 Under controlled conditions 、 data structure 、 object-oriented , Then I can't wait to talk to bug dealing . In the process , A friend has been silently accompanying us , But we never pay attention to it . He is ours import brother

import os
 Copy code 

In our code , It plays an indispensable role . But few people really understand it . Because of him ,python The power of can be brought into play , Today, let's take a good look at it .

1. Modular programming

In our real project , The amount of code may reach hundreds of thousands 、 Millions of lines . If we put these hundreds of thousands 、 Millions of lines of code are written in a file . The consequences are very serious , First, our code file will be very large , The second is to find our code through the naked eye , It's also very difficult , Let alone understand the code logic .

The idea of modular programming has emerged . Modular programming helps developers make overall planning and division of labor , And improve code flexibility and maintainability . For example, many engineers work together to develop the same system .A Logic for login function ,B Do the logic of the registration function ,C The logic of user management . These codes are separate , Inherit it into the same system through modular assembly .

Modular programming ： The code of a complete system , Split into small modules

for instance ：

（1） Non modular projects ： There is only one project py file main.py

def login():
    # Here, complete the code editing of login logic 
    pass

def register():
    #  Here, complete the coding of registration logic 
    pass

def user_manage():
    #  Here we complete the coding of user management 
    pass

if __main__ == "__main__":
    login()
    register()
    user_manage()
 Copy code 

（2） Modular projects ： Included in the project main.py、login.py、register.py、user_manage.py

• login.py

  def login():
      # Here, complete the code editing of login logic 
      pass
   Copy code 

• register.py

  def register():
      #  Here, complete the coding of registration logic 
      pass
   Copy code 

• user_manage.py

  def user_manage():
      #  Here we complete the coding of user management 
      pass
   Copy code 

• main.py

  from login import login
  from register import register
  from user_manage import user_manage
  
  if __main__ == "__main__":
      login()
      register()
      user_manage()
   Copy code 

Actually ,Python So powerful , This feature plays a big role . We don't need to write many functions ourselves , By importing modules written by others , We can use it directly , This can greatly provide efficiency . For example, we want to write a reptile , Suppose there is no modular help , We need to go from 0 Start writing ,http request 、TCP Connect 、 Return processing 、 Underlying packet encapsulation 、 analysis . But with modularity , We only need one line of code , Open the box , Is it very convenient

import requests
 Copy code 

2. Python Module in

Modular programming , So in Python in , What exactly does our module mean ？ stay Python in , We need to distinguish between several concepts ：

1. modular ： One suffix is .py The code file is a module
2. package ： One contains a lot of .py File folder .（Python3.3 Previously, this folder must contain __init__.py file ）
3. library ： It may consist of multiple packages and modules , It can be considered as the packaging of a complete project .

3. import sentence

3.1 Import from module

（1） Full import ：

Full import will import all global variables in the module 、 function 、 Classes and so on are all imported .

There are two ways to import all the contents of a module ：

import xxx

import test

print(dir(test)
#  Check what has been imported 
#  Find what we define Hello class 、hello function 、name All variables are imported 
#  And there are other things 
['Hello', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'hello', 'name']

test.hello()
print(test.name)
#  The above method needs to use ： Module name . Variable name （ Method name ） quote , Because it just introduces the whole module , The contents of the module are not introduced separately 
 Copy code 

from xxx import *

from test import *

hello()
Hello()
print(name)
#  The above can be used directly , because  from test import *  Have already put test All contents of the module are imported separately .
 Copy code 

（2） Local import

from xxx import xxx

from test import hello  #  Only from test Import in module hello function , Nothing else 
from test import hello, Hello  #  Import multiple 

hello()
 Copy code 

Set an alias for the imported content ：from xxx import xxx as yyy

from test import hello as hello_func  #  from test Import in module hello function , And set the alias to ：hello_func

hello_func()
 Copy code 

3.2 from python Import

|——test_package
	|——__init__.py
	|——test.py
	|——test2.py
|——main.py
 Copy code 

main.py：

Let's guess , Can the following two pieces of code run ？

import test_package

test_package.test.hello()

Traceback (most recent call last):
  File "/app/util-python/python-module/main.py", line 8, in <module>
    test_package.test.hello()
AttributeError: module 'test_package' has no attribute 'test'
 Copy code 

from test_package import *

test.hello()

Traceback (most recent call last):
  File "/app/util-python/python-module/main.py", line 10, in <module>
    test.hello()
AttributeError: module 'test' has no attribute 'hello'
 Copy code 

See here , Maybe there are three question marks on everyone's face ？？？ Why is it difficult to put the method of importing modules in the import package ？

In fact! , our python When importing a module , Will take our module .py Execute the file once , Then generate a module object , The functions we define in the module 、 Variable 、 Class will be added to the properties of the module object ： That's why we can pass test.hello(), because hello() yes test A property of .

What about importing packages ？ We know python Your package is essentially a folder , Folders cannot be compiled and executed . Then why can you import Well ？ actually , We are import When it comes to a bag , The execution is the... In this package __init__.py file , It can also be understood as importing packages , Just imported __init__.py

Because we __init__.py It's empty. , So we introduced a lonely .

（1） By importing modules ：

from test_package import test
test.hello()

from test_package.test import hello
hello()
 Copy code 

（2） By adding __all__ attribute ：

__init__.py file ： add to __all__ attribute

__all__ = ["test", "test2"]
 Copy code 

main.py file ：

from test_package import *

#  By means of  __init__.py It defines  __all__ attribute , When importing , You can import all the modules listed in this attribute  
test.hello()
test2.hello2()
 Copy code 

__all__ It is a convention to expose interfaces for modules , To provide ” White list “ Expose the interface in the form of . If you define __all__, Used in other documents from xxx import * When importing the file , Only import __all__ Members listed

__all__ For use only from module import * This applies .

4. Dynamic import

Above, we introduced the more mainstream import Statement import , Actually in python There are other import methods in

4.1 __import__()

__import__() Function can be used to import modules . In fact, when we use import Import Python Module time , The default call is __import__() function . It is rare to use this function directly , It is generally used to dynamically load modules .

__import__(name, globals=None, locals=None, fromlist=(), level=0) Parameters ：

• name： Module name to import , Available variables
• globals and locals： Default values are usually used . Using the given globals and locals Variable to determine how to resolve in a package context name
• fromlist： Specify the name of the sub module or object to import , They are imported from the module by name
• leve： Specify how to import the module .level by 0 Absolutely import ; level A positive value indicates relative to the call __import __() Module directory , Number of parent directories to search

os_obj = __import__("os")

print(os_obj.getcwd())
 Copy code 

os_obj = __import__("test_package.test")

os_obj.test.hello()
 Copy code 

4.2 importlib

importlib yes Python A standard library in ,importlib It can provide a very comprehensive function

import importlib
myos=importlib.import_module("os")
myos.getcwd()
 Copy code 

4.3 A usage scenario

A timed task scenario , There are many such scheduled tasks stored in the database ：cmdb.tasks.get_host_info, It means calling cmdb Under bag tasks Under the module of get_host_info function . How do we achieve , Through a string in this format , To call the corresponding function ？

You can think about how to do .

def exec_task(task_name):
	if not task_name:
		return -1, "task name must not None"
	try:
    	module_name = task_name.rsplit(".", 1)[0]
        method_name = task_name.rsplit(".", 1)[1]
        
        #  Dynamic import tasks modular 
        module_obj = __import__(module_name)
        if not hasattr(module_obj, method_name):
			return -1, "function not found"
    except:
    	return -1, "has Error"
        
    task = getattr(module_obj, method_name)
    task()
 Copy code 

5. Search path

I wonder if you have ever thought about such a problem , When we import a module or a package ,python Where did you find this module ？

Python The path of the search module is composed of four parts ：

• The main directory of the program 、
• PATHONPATH Catalog
• Standard link library Directory
• .pth File directory ,

The paths of these four parts are stored in sys.path In the list .

pth file ：pth Files are used to add additional sys.path namely python Search path , Generally in github The package downloaded on will have a setup.py, The execution of this file will occur in （ At present python In the environment site-packages Folder generation ） One .pth file

import sys
print(sys.path)

[
    '/app/util-python/python-module',    #  The directory where the current program is located  
    '/usr/lib/python37.zip', 
    '/usr/lib/python3.7', 
    '/usr/lib/python3.7/lib-dynload', 
    '/app/python-virtualenv/aioms-env/lib/python3.7/site-packages'  #  Download the third-party package directory 
]
 Copy code 

When our When importing a package ,python The interpreter searches these directories in turn . If not found in these directories , The program will report an error

Suppose there's a file ： /app/test_pack.py

Our current program path is ：/app/util-python/python-module/main.py

import test_pack

Traceback (most recent call last):
  File "/app/util-python/python-module/main.py", line 5, in <module>
    import test_pack
ModuleNotFoundError: No module named 'test_pack'
 Copy code 

There was no accident , Wrong report , I can't find this bag

import sys
sys.path.append('/app')

import test_pack

test_pack.hello()
#  Test import custom path 

print(sys.path)
[
    '/app/util-python/python-module',
    '/usr/lib/python37.zip', 
    '/usr/lib/python3.7', 
    '/usr/lib/python3.7/lib-dynload', 
    '/app/python-virtualenv/aioms-env/lib/python3.7/site-packages', 
    '/app' #  We found an additional search directory , It found our... In this directory test_pack.py
]
 Copy code 

therefore ： The search paths of modules and packages are not fixed , We can customize it , Of course, the above method is only temporary

therefore , When this error occurs in our program ：ModuleNotFoundError: No module named 'test_pack'

Problem solving trilogy ：

1. Check where the path of the downloaded and installed package is
2. Use sys.path have a look , Is the downloaded package here

6. Relative Import and absolute import

Suppose the package structure ：

main.py
packageA/
    __init__.py
    moduleA.py
    moduleB.py
packageB/
	__init__.py
	moduleA.py
	moduleB.py
 Copy code 

about packageA/moduleA.py：

Absolutely import ： All modules import From the “ The root node ” Start . The location of the root node is determined by sys.path The path in determines

from packageA import moduleB
 Copy code 

Relative Import ： Only care about the module location relative to your current directory

• . Current directory peer lookup
• .. Find the parent of the current directory

from . import moduleB
 Copy code 

7. Cross reference （ Import loop ）

What is cross import , The two packages are imported into each other

main.py
package/
    __init__.py
    moduleA.py
    moduleB.py
 Copy code 

from . import moduleB
 Copy code 

I don't want to expand too much here , It is recommended that everyone use absolute import to finish ！ So as not to dig a hole for yourself

moduleA.py:

from moduleB import hello_b
 Copy code 

moduleB.py:

from moduleA import hello_a
 Copy code 

In this ： modular A Called module B A method of 、 And modules B Also called the module A A method of , This is called cross import

First, let's see what problems this will lead to

ImportError: cannot import name 'hello_b' from 'moduleB'
 Copy code 

This will throw an exception . The reason for the abnormality is that ： When importing ,moduleA Need to access moduleB Of hello_b, But no hello_b It's not initialized yet . So an exception will be thrown

Generally speaking ： This is due to large Python In Engineering , Improper architecture design , Mutual reference between modules

terms of settlement ：

1. Structure optimization , Dereference
2. import The statement is placed at the end of the module
3. import Statement placed in a function

8. Install third party package

What is a third-party package ？ There is a popular saying in the programming circle ：“ Don't make wheels over and over again ！”. The wheel here is actually our third-party package . When we want to make a car , We can just use the existing parts to assemble them . There's no need to start from 0 Start , To build the wheels 、 Engine, etc . Because others have made it .

In the computer industry , Someone else built the wheel , There must always be a place to save , It can be seen and used by other users . This requires an authoritative tripartite organization to manage these wheels .

PyPI It's such a role , PyPI(Python Package Index) yes python The official third-party warehouse , Everyone can download third-party libraries or upload their own developed libraries to PyPI.

How to manage these three-party packages ？ There are many methods on the market . But the most widely used is pip

pip It's a modern , General purpose Python Package management tools , The tool provides the right Python Bag search 、 download 、 install 、 Uninstalled features

7.1 pip install

#  see pip edition , Can be judged pip Whether to install 
pip -v
 Copy code 

In general , We downloaded it from the official website python After the installation , I will bring this tool with me , We don't have to worry too much , however , If it's too bad , Yours Python There happens to be no pip This tool , What shall I do? ？

about linux user ：

$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py #  Download the installation script 
$ sudo python get-pip.py #  Run the installation script 
 Copy code 

sudo apt-get install python-pip
 Copy code 

window Users are similar to , Just go to the official website to download pip Installation package , Reuse python Can be installed

7.2 Install third party package

pip install Django  #  Install the latest version 
pip install Django==1.0.4  #  Specify the version 
pip install Django>=1.0.1  #  Specify the minimum version 
pip install Django<=1.0.1  #  Specify the maximum version 
 Copy code 

because PyPI The image source is foreign , Sometimes the download is very slow , At this time, we can use the domestic image source

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple Django
 Copy code 

Domestic image source address ：

• tsinghua ：pypi.tuna.tsinghua.edu.cn/simple
• Alibaba cloud ：mirrors.aliyun.com/pypi/simple…
• douban ：pypi.douban.com/simple/
• ...... There are others to search for

7.3 Other operating

pip uninstall Django  #  Uninstall installed Libraries 
pip install --upgrade Djano==2.0.1  #  Upgrade installed packages 
pip list  #  List installed libraries 
pip freeze > requirements.txt  #  Add the current project dependency to the exported text file 
pip install -r requirements.txt  #  Install according to the dependencies in the text file exported above 
 Copy code 

Pay attention to the official account of Xiaobian ： Learn secretly , Kill them