for 循環 output 進入單個列表

Question

我正在嘗試提取一些包含其他鏈接的鏈接，並努力將 output 從 for 循環中獲取到單個列表中，如下所述。

我的代碼：

pages = ['https://pagetoscrape.com/?page=1',
         'https://pagetoscrape.com/?page=2',
         'https://pagetoscrape.com/?page=3'
        ]

for u in pages:
    response = requests.get(u)
    data = response.content
    soup = BeautifulSoup(data, 'lxml')
    for links in soup.find_all('div', class_='item-to-scrape'):
        link = links.a['href']

        print(link)

Output：

pagetoscrape.com/url1
pagetoscrape.com/url2
pagetoscrape.com/url3
pagetoscrape.com/url4
pagetoscrape.com/url5
pagetoscrape.com/url6
pagetoscrape.com/url7
...

如何獲得這樣的列表以便稍后將其用於類似於“pages”數組變量的其他操作（另一個循環選擇鏈接）？：

['pagetoscrape.com/url1', 'pagetoscrape.com/url2', 'pagetoscrape.com/url3', 'pagetoscrape.com/url4', ...]

Answer 1

使用列表理解構建您的列表：

link_list = [links.a['href']
                for links in soup.find_all('div', class_='item-to-scrape') ] 

Answer 2

如果我理解正確，你想要這樣的東西。

pages = ['https://pagetoscrape.com/?page=1',
         'https://pagetoscrape.com/?page=2',
         'https://pagetoscrape.com/?page=3'
        ]

urls = []

for u in pages:
    response = requests.get(u)
    data = response.content
    soup = BeautifulSoup(data, 'lxml')
    for links in soup.find_all('div', class_='item-to-scrape'):
        link = links.a['href']
        urls.append(link)
        print(link)