How can I make this code run better/faster (threading or multiprocessing)? And how can it be done?

Question

I have a simple bot that logs into a site with cookies and checks for the price of an item and if that price meets the price I set it to, it purchases the item.

I am looking for ways to improve the speed of this bot. I really do not know if multiprocessing will make this bot faster in this case.

I am also looking for ways to make it more efficient if any.

session = requests.session()
session.cookies["cookie"] = ""

log_in = session.get("https://www.example.com")
if log_in.status_code == 200:
    print("Logged In")
else:
    raise ValueError("Invalid Cookie")

crsf_token = ""

def token():
    global crsf_token
    while True:
        crsf_token = re.search(r"<script>XsrfToken.setToken\('(.*?)'\);</script>", session.get('https://www.example.com').text).group(1)
        time.sleep(5)

def _cthread():
    while True:
        try:
            req = session.get(f"https://www.example.com/productID")
            if req.status_code == 429:
                time.sleep(5)
                continue

            for i in req.json()["data"]["Sellers"]:
                if i["Price"] <= 300:
                    session.post(f'https://www.example.com&expectedPrice={i["Price"]}&expectedSellerID={i["SellerId"]}&userAssetID={i["UserAssetId"]}', headers={"X-CSRF-TOKEN": crsf_token})
        except requests.urllib3.exceptions.ConnectTimeoutError as E:
            pass


while True:
    threading.Thread(target=_cthread).start()
    threading.Thread(target=token).start()

I have not gotten much success with this but it does work right now.

Answer 1

尝试从函数token()和_cthread()删除while True

Answer 2

Assuming you're not bandwidth constrained and the site won't shut down too many POST s submitted at once, you might gain a little from threading your post calls, changing:

        for i in req.json()["data"]["Sellers"]:
            if i["Price"] <= 300:
                session.post(f'https://www.example.com&expectedPrice={i["Price"]}&expectedSellerID={i["SellerId"]}&userAssetID={i["UserAssetId"]}', headers={"X-CSRF-TOKEN": crsf_token})

to:

        allposts = [f'https://www.example.com&expectedPrice={i["Price"]}&expectedSellerID={i["SellerId"]}&userAssetID={i["UserAssetId"]}'
                    for i in req.json()["data"]["Sellers"] if i["Price"] <= 300]
        if allposts:
            with Pool() as pool:
                pool.map(partial(session.post, headers={"X-CSRF-TOKEN": crsf_token}), allposts)

adding the following imports to the top of your file:

from multiprocessing.dummy import Pool  # Gets thread based worker Pool class
from functools import partial           # Lets you bind headers up front so you only need to pass URL to post

To avoid excessive thread spawning, you might create the pool outside the loop in that function, rather than creating it on demand only when allposts is non-empty.

I'd also recommend removing the while True: from the top level of your code; token and _cthread are already both infinite loops, so having both loops means spawning an infinite number of threads, each of which runs forever, when you really only need two persistent threads.