for 循环 output 进入单个列表

Question

我正在尝试提取一些包含其他链接的链接，并努力将 output 从 for 循环中获取到单个列表中，如下所述。

我的代码：

pages = ['https://pagetoscrape.com/?page=1',
         'https://pagetoscrape.com/?page=2',
         'https://pagetoscrape.com/?page=3'
        ]

for u in pages:
    response = requests.get(u)
    data = response.content
    soup = BeautifulSoup(data, 'lxml')
    for links in soup.find_all('div', class_='item-to-scrape'):
        link = links.a['href']

        print(link)

Output：

pagetoscrape.com/url1
pagetoscrape.com/url2
pagetoscrape.com/url3
pagetoscrape.com/url4
pagetoscrape.com/url5
pagetoscrape.com/url6
pagetoscrape.com/url7
...

如何获得这样的列表以便稍后将其用于类似于“pages”数组变量的其他操作（另一个循环选择链接）？：

['pagetoscrape.com/url1', 'pagetoscrape.com/url2', 'pagetoscrape.com/url3', 'pagetoscrape.com/url4', ...]

Answer 1

使用列表理解构建您的列表：

link_list = [links.a['href']
                for links in soup.find_all('div', class_='item-to-scrape') ] 

Answer 2

如果我理解正确，你想要这样的东西。

pages = ['https://pagetoscrape.com/?page=1',
         'https://pagetoscrape.com/?page=2',
         'https://pagetoscrape.com/?page=3'
        ]

urls = []

for u in pages:
    response = requests.get(u)
    data = response.content
    soup = BeautifulSoup(data, 'lxml')
    for links in soup.find_all('div', class_='item-to-scrape'):
        link = links.a['href']
        urls.append(link)
        print(link)