[Python automated operation and maintenance road] path inventory

In fact, we are more concerned about , What can you learn after reading the author's article , What can be done to come out , Or we are meeting a real demand , You need to click on the relevant skill tree , So let me give you a general introduction of what you can learn from me .

The choice of path

• The first is for those who are engaged in operation and maintenance , But I met some readers who need automation in my work , So the author here does not advocate that we build the whole automatic operation and maintenance system , One is not necessary , Another is that it may also consume considerable energy , So we just click the skill tree related to our actual work for the time being .

• But some readers want to seek a breakthrough in career development , It is planned to transform from traditional operation and maintenance to automatic operation and maintenance , Or friends who plan to start their career from automatic operation and maintenance at the beginning , Then you need to go through some more detailed and systematic learning .

Path planning

So for automatic operation and maintenance, the first programming language is Python, No matter from Python In terms of the richness and difficulty of the whole Ecology , They are the best language for us to try automated operation and maintenance . But online for Python A lot of language tutorials , Why should I explain it here ？ This is actually equivalent to another problem ： Whether new programmers can get started quickly by watching similar tutorials Python?, Or easy to understand ： Can you understand ?

Most friends engaged in operation and maintenance may not have programming experience , I haven't systematically studied the underlying principles of computer , So the standard tutorial will feel , It seems that , But there was no plenary session ; It seems that , But I can't use , So I graduated as a computer programmer , According to their own learning methods and the learning experience of their friends , Let's get started quickly through some vivid examples and actual scenes Python This language , And can be directly applied to practical work .

【 Automation operation and maintenance novice village 】

The content of novice village is to build a simple website with pages for everyone CMDB, This is also in line with the actual needs of many o & M friends , For example, I want to make a small function , There is another simple page for others to click on . In addition, some knowledge points involved will also be deeply excavated in detail , Presented in an external chapter , Let readers who are interested in it know more .

• Python Basic data structure and operation

  take CMDB Add, delete, modify, check and Python Combination of data structures , Explain the boring basic knowledge in simple terms

  • Off the coast ： Detailed explanation of data structure
  • Off the coast ： Detailed explanation of string
  • Off the coast ： Detailed explanation of array
  • Off the coast ： A detailed explanation of the dictionary
  • Outside the interview ： Algorithm

• Python Functions and objects

  How to build CMDB Gradually disassemble the functions and objects of

  • Off the coast ： Deep understanding of object-oriented
  • Off the coast ： Introduction to design mode

• Flask Web frame

  Will be right CMDB Operation and Web Frame combination , By means of a page CMDB To operate

  • Off the coast ： basic html and css Explain
  • Outside the interview ： computer network

• MySQL database

  take CMDB Store in database , The addition, deletion, modification and query are transferred from file reading and writing to database operation

  • Off the coast ：MongoDB database
  • Outside the interview ： database

• Linux Deploy the application

  Deploy the application in Linux Stable operation on the system

  • Outside the interview ： operating system

【 Primary village of automatic operation and maintenance 】

In fact, the design of the content sequence of the primary village is similar to that of the novice village , Finally, I will make an application with front-end pages , But what this part uses Python Knowledge will go a little deeper , So the suggestion has been right Python Have a certain understanding of friends , You can skip the novice village and look directly at the content of the primary village , Because the primary village will be a demand scenario often encountered by more o & M friends , It is also the road that must be taken to do automated transportation .

• paramiko and netmiko

  The usage and advantages and disadvantages of the two

• Python Regular parsing

  Will pass through SSH Regular analysis of all kinds of information collected

  • Off the coast ：text-fsm analysis

• Python snmp

  How to use Python call SNMP Acquisition command , And explain OID Usage mode

• NetConf

  How to use NetConf Operate on network devices

• Crontab And callback CMDB

  How to set scheduled execution SSH Conduct regular patrol inspection for tasks , Update and write back CMDB

  • Off the coast ：Python scheduler

• Flask Web frame

  take SSH and NetConf And Web Frame combination , adopt API Method or the front end calls its execution

【 Automation operation and maintenance intermediate village 】

The biggest difference between intermediate village and primary village is , From the completion of simple requirements application , Design and implementation of a more complex architecture , And scenarios that can support larger operation and maintenance .

• Celery Task queue

  How to use task queues to process a large number of tasks asynchronously SSH or NetConf Perform the operation

  • Off the coast ：Redis database
  • Off the coast ：RabbitMQ middleware

• Rsyslog Log collection

  adopt Rsyslog Collect equipment logs and store them for display and query .

• Mysql performance optimization

  Support storage and query of larger amount of data , Optimize the database , Sub database and sub table

• Fore and aft end separation

  Due to the rich functions of the system , By the original front and rear end not separated system , Split into front and rear end separated applications .

• Vue Front end explanation

  Use Vue As the front-end framework of the system , Explain Vue Practical operation and more front-end knowledge .

【 Automation operation and maintenance advanced village 】

If you want to make a complete automatic operation and maintenance system , It is necessary to have a corresponding and perfect concept and methodology of automatic operation and maintenance , Therefore, the senior village will design more introduction to the concept of automatic operation and maintenance , As well as more complex and adaptive automated operation and maintenance systems for larger enterprises .

• Celery Time sharing task queue

  Use Celery Task queue , take SNMP A large number of acquisition tasks or SSH Change tasks are published in different queues according to frequency for efficient collection

• Flink Handle

  Use Flink Yes SNMP The collected data shall be cleaned and stored

• Rsyslog - Kafka - Consumer- MySQL

  How to collect and parse the logs reported by the device , And pass Kafka Middleware peak shaving , And store it

  • Off the coast ： Kafka middleware

• ELK, Rsyslog - Kafkf - LogStash - ElasticSearch - Kibana

  Use ELK The suite stores and displays logs

• Warning notice , shielding , Inhibition

  How to set alarm notification , Shielding and suppression mechanisms

• Task arrangement

  Register the change step as an execution operator through the white screen , And issue after arrangement

• Intention based packet configuration distribution

  Group devices according to roles , And set intent based template commands , Use SSH or NetConf The method is combined with it to change or query the network equipment

Summary of the path

The above path combing is the most systematic practice of automatic operation and maintenance that I can give you at present , In the process of updating , I may update and iterate the content according to the needs of readers or the thinking summary in my daily work , So the path is not fixed , It's just a learning direction , And the process of learning is bound to have obstacles and difficulties , I hope everyone can have a firm determination to overcome the difficulties one by one , Of course, it will also provide groups for everyone to learn and communicate , For everyone to discuss , Make progress with each other , You are also welcome to consult me if you have any questions . Finally, I hope everyone can get their own results .