current position:Home>Troubleshooting ideas and summary of Django connecting redis cluster

Troubleshooting ideas and summary of Django connecting redis cluster

2022-02-01 17:21:07 Pear cauliflower

The problem background

  • Django The project is released to the production environment for the first time , After startup, I found that I couldn't connect Redis colony .
  • Test environment redis The cluster has no password , The configuration format of test environment production will be different
  • Used in the production environment redis To configure , No one has verified it before , I don't know whether it's a single machine or a cluster
  • Production environment redis Special match in password @, Colleagues said they might Django Cache Configured URI Cutting symbol conflict , He was before celery broker_url This pit is encountered during configuration .

Investigation thought

  • The configuration of the test environment is different from that of the production environment , So be extra careful , The configuration of the production environment cannot copy the configuration of the test environment
  • Give priority to official redis-cli determine redis Cluster or client , Then modify the configuration related to the code
  • The production environment cannot be connected locally , Because you need to log in to the production environment through the springboard machine , Direct use of Django shell debugging , Instead of resubmitting every time you modify the code , And then build .
  • When something goes wrong , First doubt your code problems , Instead of looking for the problem of the framework itself , In especial Django This big framework .

The screening process

Confirm the client type

  • Make one of the simplest set Command test , Direct error .
rhea-flaskapi-live-sg(sg2|10.xxx:40681)@/workspace$ redis-cli -h redis.xxx.shopee.io -p 10010 -a [email protected]
redis.xxx.shopee.io:10010> set a 123
(error) MOVED 15495 10.xxx.104:10011
 Copy code 
  • After search engine query , Confirm this redis It's clustering , The cluster connection command is redis-cli -c -h Connection address -p port -a password There are fewer commands above -c Parameters

start-up Django Shell Connect redis

from django_redis import get_redis_connection
conn = get_redis_connection()
 Copy code 

Directly on a connection failure

D0F23F45D43E899E71F1B54B84571D04

image-20211130111929339

After confirming that the account, password and port are OK , Question: has the colleague configuration been verified , He said directly from the Internet cv Of , Completely unverified . Pit !!!

Retest after modifying the configuration

Compare the production configuration with the test configuration , Revised several places

  • LOCATION from str Change to List[str], Add default db, That is to say [ f'redis://{DEFAULT_REDIS_URL}:{DEFAULT_REDIS_PORT}/0']
  • REDIS_CLIENT_CLASS': 'rediscluster.RedisCluster This cluster connection requires clients
  • Change the connection pool configuration to 'CONNECTION_POOL_CLASS': 'rediscluster.connection.ClusterConnectionPool'

Full of confidence , Restart django shell test , As a result, the connection is still not ! At this time, the mood began to be a little bad ~

calm ,django shell no way , The use python shell Try connecting directly ?

Python Shell Direct connection redis

image-20211130113526742

There is nothing wrong with it , Directly connected to !

A face of meng , What's the problem !

Don't lose heart , Repeatedly modify the configuration , test Django Shell Connect Redis

The result is still not connected .

Thinking in the process of going home from work

Unknowingly, it's 9 p.m , Tired , I don't want to roll . Go home from work

On the way home, the whole brain is troubled by this problem . Does the password contain @ The symbol of redis colony ,Django Really can't connect ? Ask yourself repeatedly .

Asked other colleagues , Are there any other problems in the production environment redis Clusters can be used to debug . unfortunately , did not .

or , I created one myself redis colony , Set the password to contain @ Symbol ?

But , Create your own local redis Cluster is so troublesome . To install the virtual machine locally , Think of a bunch of configurations and directly persuade them to retreat .

The tangle after returning home

After taking a bath , I talked to my wife 1h Left and right video . It has arrived 11 Some more , Get ready for sleep ?

That's impossible , With problems, it's hard to fall asleep ! Ah , This is an old problem .

Suddenly thought of a point , The principle of minimum trial and error . It's troublesome to build local clusters by yourself , The company has no redundant clusters .

Then buy a cloud version directly redis colony ? Do as you say , Just get out of bed , Turn on the computer .

The principle of minimum trial and error , Buy cloud Redis colony , Instead of building it yourself

Then the problem comes again , Alibaba cloud or Tencent cloud ?

In view of the double 11 Bought Tencent cloud 2c 4g 8m Server for , as long as 199 can 3 year .

Compare with the one who bought Alibaba cloud before 1c 2g 1m The server ,3 Years also have to 100 many .

Instantly feel good about Tencent cloud , Decided to buy Tencent cloud first .

Buy Tencent cloud redis colony

A operation , Found that Tencent cloud is really difficult to use :

  • Select and configure the purchase page , After submission , Because I have no balance , Prompt me to recharge . When I recharge , As a result, the previously selected configuration is gone , Have to choose again .
  • redis The cluster has no restart function ???
  • When setting up a security group , The instance can only be bound on the page of the security group ; Cannot bind security group on instance page .

The most important , After binding the instance to the security group , The Internet is still inaccessible ???( No matter the , Anyway, I'm just angry )

Buy Alibaba cloud redis colony

I'm so disappointed with Tencent cloud , I have to put the last straw on Alibaba cloud .

Fortunately , Alibaba cloud didn't let me down !

Click, click, click , A operation :

  • Create examples , Alibaba cloud is much slower than Tencent cloud , Probably 7min about ( Pure feeling )

  • Configure Internet access , Very convenient . This operation logic is simply killing Tencent cloud .

    image-20211130120920961

  • Configure white list ,redis-cli Connect the test , Successfully passed !

    image-20211130121327756

  • Modify the configuration , start-up Django shell test .

  • image-20211130121629842

image-20211130121524863

The password contains @ Symbol , But there's nothing wrong with the connection !!!

thus , The problem is finally solved !!!

I can't wait to go to the company tomorrow to verify , But looking back , It's already 1:30 late at night .

He said to himself :" Go to sleep. , Roll King "

Go to the company the next day to verify

rhea-flaskapi-live-sg(sg2|xxxx:26863)@/workspace$ python3.8 manage.py shell --settings=rhea.settings.prod
rhea.settings.prod
Python 3.8.7 (default, Oct  1 2021, 14:58:33)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django_redis import get_redis_connection
>>> conn = get_redis_connection()
>>>>conn.get("rhea:access_token:abc")
'[email protected]'

 Copy code 

After comparison , The discovery configuration only needs the production configuration, and only needs to add one more... To the tested configuration :

image-20211130131601844

Repair the most grinding bug, Often only a small change is needed ~

Why did the test environment report no errors ???

Because of the test environment redis The cluster does not require a password

summary

  • All code that has not been verified by itself , Be sure to use it carefully

  • The principle of minimum trial and error . When you want to verify something , Try to use the least cost to verify

  • Try to keep the configuration of production and test environments the same

copyright notice
author[Pear cauliflower],Please bring the original link to reprint, thank you.
https://en.pythonmana.com/2022/02/202202011721025522.html

Random recommended