使用webscrapig对网站进行压力测试

Question

My friend wrote a website by node.js, and I want to help him to do the stress test just in case it crashed some day. I used beautifulsoup to test it, and it went well.

Code:

for i in range(0,1000):
    #i=i+1
    l=randint(2008, 2018)
    first="year=%d" %l
    k=randint(1600, 2400)
    second="cc=%d" %k
    url="http://xx.xxx.xxx.xxx:xxxx/outputData?{0}&{1}&submit=Submit".format(first,second)
    res = requests.get(url)
    soup=BeautifulSoup(res.text,'lxml')
    print(soup) 

Instead of running 1000 times one by one in order, is there any other way to test it if I want to run it at the same time with 1000 codes by using python? Thanks!

Answer 1

You can do it with threading . But I don't recommend to create 1000 thread and run it at the same time.

import threading
def testit():
    l=randint(2008, 2018)
    first="year=%d" %l
    k=randint(1600, 2400)
    second="cc=%d" %k
    url="http://xx.xxx.xxx.xxx:xxxx/outputData?{0}&{1}&submit=Submit".format(first,second)
    res = requests.get(url)
    soup=BeautifulSoup(res.text,'lxml')
    print(soup) 

threads = [threading.Thread(target=testit) for i in range(1000)] # Not recommend to use 1000 here

for t in thread:
    t.start()

Answer 2

Use threads. Open up n threads, and ping the server at the same time. You can look into https://www.python-course.eu/threads.php

But running threads is a CPU intensive task, so make sure you keep n controlled. Giving a higher value to n , could stress the client.