如何给每个 URL 自己的线程

Question

I have been working on a small PoC where I have been trying to improve my knowledge with Threading but unfortunately I got stuck and here I am,

import time

found_products = []

site_catalog = [
    "https://www.graffitishop.net/Sneakers",
    "https://www.graffitishop.net/T-shirts",
    "https://www.graffitishop.net/Sweatshirts",
    "https://www.graffitishop.net/Shirts"
]


def threading_feeds():
    # Create own thread for each URL as we want to run concurrent
    for get_links in site_catalog:
        monitor_feed(link=get_links)


def monitor_feed(link: str) -> None:
    old_payload = product_data(...)

    while True:
        new_payload = product_data(...)

        if old_payload != new_payload:
            for links in new_payload:
                if links not in found_products:
                    logger.info(f'Detected new link -> {found_link} | From -> {link}')
                    # Execute new thread as we don't want to block this function wait for filtering() to be done before continue
                    filtering(link=found_link)

        else:
            logger.info("Nothing new")
            time.sleep(60)
            continue


def filtering(found_link):
    ...

1 - Im currently trying to do a monitor where I have multiple links to start with, my plan is that I would like to each url to run concurrently instead of needing to wait 1 by 1:

def threading_feeds():
   # Create own thread for each URL as we want to run concurrent
   for get_links in site_catalog:
      monitor_feed(link=get_links)

How can I do that?

---

2 - If we seem to find a new product that has appeared to the given URL inside the monitor_feed , how can I set a new thread to execute the call filtering(link=found_link) ? I dont want to wait for it to be done before it continues to loop back for the While True but instead it should do the filtering(link=found_link) in the background while it stil executes the monitor_feed

Answer 1

import concurrent.futures    
with concurrent.futures.ThreadPoolExecutor() as executor:
            executor.map(monitor_feed, site_catalog)

You can use ThreadPoolExecutor .