current position:Home>Python tiktok programming analysis and case mitmproxy + python+Appium grab shake fan information

Python tiktok programming analysis and case mitmproxy + python+Appium grab shake fan information

2022-02-02 10:55:36 cwh5920

Preface

  1. Preparation of the environment

  2. Environment configuration
    2.1 filder To configure
    2.2 xposed and JustTrustMe Configuration of
    2.3 mitmproxy To configure
    2.4 To configure Android Studio
    JDK Configuration of
    To configure Appium

  3. Locate page elements

  4. Run the project
    Preface

  5. Preparation of the environment
    Night God Simulator (Android 5.1.1)
    filder
    mitmproxy
    python3.7.3
    jdk1.8.0_251
    Android Studio(Android SDK 5.1)
    xposed(Android 5.1)
    JustTrustMe
    Let's start with , Tiktok ssl verification , Night God simulator installation xposed and JustTrustMe after , To capture packets normally , Tiktok is not open. .

  6. Environment configuration
    except python3.7.3 and mitmproxy, The rest of the resources are in Baidu online disk , Self taking , Address :https://pan.baidu.com/s/1ZtvZIQvQ8A6rp02HtAOudQ, Extraction code :4mu1.

2.1 filder To configure
Install well filder after , open filder, top left corner Tools -> Options, Set as follows :

Click on Actions, choice Trust Root Certificate, Then choose all yes , install window Certificate for

To this step filder,PC The end is well matched , Then turn off the domain firewall , obtain PC Terminal ip Address , The method is as follows :

Then turn on the night God simulator , Get into wifi, Configure agent :

Then open the web page to enter 192.168.1.19:8888, Download the certificate ,

Download complete , Enter settings -> Security -> from SD Card installation , Install the certificate you just downloaded , Name yourself ,

everywhere ,filder You can catch the packets on the night God simulator , And tiktok must be installed. xposed and JustTrustMe.

2.2 xposed and JustTrustMe Configuration of
Put it in the network disk xposed and JustTrustMe Of apk After the document , Directly click the installation on the right side of the night God APK, Find the of these two frameworks apk file , Click on the install , Here's a sequence , Install first xposed, After installation, the night God desktop has a xposed install, open , Then download xposed,

It will be displayed after downloading Xposed frame 89 Version activated , Then install JustTrustMe Of apk, After installed , stay xposed install Check inside JustTrustMe,

Installation is complete here , Tiktok and download the latest version of the jitter. , You can use it filder Grab the bag. :

Write this down url First half of , It will be useful later .

2.3 mitmproxy To configure
First , stay PC End pass pip install mitmproxy install mitmproxy modular , After installed , Continue to pass cmd, Input mitmproxy, This will be displayed in the user directory of the system disk .mitmproxy Many certificates are generated under the folder , Click on mitmproxy-ca-cert.p12, Select local computer , Go straight to the next step , The location of the password is not required , Go straight to the next step , The certificate is shown in the figure below :

such PC The end certificate is ready , Then match the night God , take mitmproxy-ca-cert.pem Drag directly into the night God simulator , Or install as above filder Certificate operation method of night God simulator , Installation certificate , Then create douyin_fans.py file :

import json

def response(flow):
if ‘api3-normal-c-hl.amemv.com/aweme/v1/user/follower/list’ in flow.request.url:
for user in json.loads(flow.response.text)[‘followers’]:
user_info = dict()
user_info[‘nickname’] = user[‘nickname’]
user_info[‘share_id’] = user[‘uid’]
user_info[‘douyin_id’] = user[‘short_id’]
user_info[‘sec_uid’] = user[‘sec_uid’]
if user_info[‘douyin_id’] == ‘0’:
user_info[‘douyin_id’] = user[‘unique_id’]
user_info[‘update_time’] = user[‘unique_id_modify_time’]
print(user_info)
Then configure the night God wifi Agent for , alike 8888 Port and ip, Turn off domain firewall , And then in douyin_fans.py File directory open command line input mitmdump -p 8888 -s douyin_fans.py, And then turn on the night's tiktok and go into the fans. , You can see the output on the command line , Here's the picture :

hither ,mitmproxy It's already configured , And you can catch fans , However, you need to manually slide the fan list to turn the page , This is where we need to use it Appium Auto slide fan list .

2.4 To configure Android Studio
Let's explain why we download it first Android Studio, Mainly the night God simulator is Android System , While using Appium control Android The system needs Android Of SDK, Now the Android SDK And Android Studio Is bound. , Install well Android Studio after , Follow the diagram below :

Get into SDK Manager, install Android5.1, And remember skd Path :

Install well SDK after , Enter the corresponding path , Then open the environment variable configuration of the system ANDROID_HOME As well as some path:

And then platform-tools Under the adb.exe Put the file in the night God simulator bin Directory , And nox_adb.exe File replacement , And change to the same name nox_adb.exe, Here we are Android SDK The environment is fully equipped .

JDK Configuration of
Install well jdk after , Configure... In the system variable JAVA_HOME, And in path Add jdk Of bin route ,jdk The environment is perfect .

To configure Appium
Install well Appium Open directly after , Start the service :

platformName: Platform name , It's usually Android or iOS.

deviceName: Equipment name , Specific types of mobile phones

appPackage:App Package name

appActivity: entrance Activity name , Usually, the . start

adb shell

dumpsys window |grep mCurrentFocus

Then click on the magnifying glass , Then configure the following parameters according to the figure :

then save, then start session, So night God tiktok will automatically open. , All environments have been configured , Let's start the related work of page element positioning and crawling .

  1. Locate page elements
    open cmd Input monitor, An interface will appear later , And then open the tiktok. , Do the following :

Operate from left to right , In this way, you can get the image of the page element magnifying glass id, The following account search click elements use the same method to get the information of all elements step by step id value , To write douyin_appium.py file :

from appium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
import time

option = {
“platformName”: “Android”,
“platformVersion”: “Andriod5.1.1”,
“deviceName”: “127.0.0.1:62001”,
“appPackage”: “com.ss.android.ugc.aweme”,
“appActivity”: “com.ss.android.ugc.aweme.main.MainActivity”,
“noReset”: True,
“unicodekeyboard”: True,
“resetkeyboard”: True,
“automationName”: “UiAutomator1”
}

driver = webdriver.Remote(‘http://localhost:4723/wd/hub’, option)

try:
if WebDriverWait(driver, 5).until(lambda x: x.find_element_by_id(‘com.ss.android.ugc.aweme:id/c63’)):
driver.find_element_by_id(‘com.ss.android.ugc.aweme:id/c63’).click()
except:
pass

def get_size():
x = driver.get_window_size()[‘width’]
y = driver.get_window_size()[‘height’]
return (x, y)

try:
if WebDriverWait(driver, 3).until(lambda x:x.find_element_by_id(‘com.ss.android.ugc.aweme:id/ai4’)):
driver.find_element_by_id(‘com.ss.android.ugc.aweme:id/ai4’).click()
driver.find_element_by_id(‘com.ss.android.ugc.aweme:id/ai4’).send_keys(‘lv123052’)
if WebDriverWait(driver, 3).until(lambda x: x.find_element_by_id(‘com.ss.android.ugc.aweme:id/ghv’)):
driver.find_element_by_id(‘com.ss.android.ugc.aweme:id/ghv’).click()
if WebDriverWait(driver, 2).until(lambda x: x.find_element_by_id(‘android:id/text1’)):
driver.find_element_by_id(‘android:id/text1’).click()
driver.find_element_by_id(‘com.ss.android.ugc.aweme:id/ckr’).click()
except:
pass

try:
if WebDriverWait(driver, 5).until(lambda x:x.find_element_by_id(‘com.ss.android.ugc.aweme:id/bce’)):
driver.find_element_by_id(‘com.ss.android.ugc.aweme:id/bce’).click()
time.sleep(2)
except:
pass

size = get_size()
x1 = int(size[0]*0.5)
x2 = int(size[0]*0.7)
y1 = int(size[1]*0.9)
y2 = int(size[1]*0.2)

while(True):
time.sleep(0.5)
driver.swipe(x1, y1, x2, y2)

  1. Run the project
    Nocturnal agent open , Domain Firewall OFF , And then use mitmdump The command runs douyin_fans.py file , It has to be explained here ,mongdb Database operation in douyin_fans.py Self adding in file , And then directly run,douyin_appium.py file , This will crawl tiktok's account information. , The effect is as shown in the picture :

————————————————

copyright notice
author[cwh5920],Please bring the original link to reprint, thank you.
https://en.pythonmana.com/2022/02/202202021055035384.html

Random recommended