如何從網站的所有頁面中抓取數據並將其寫入文件？

Question

我正在編寫一個腳本來搜索網站上所有玩家的姓名和 OVR。 當我嘗試使用多頁執行此操作並將結果寫入文件時，output 僅來自最后一頁。

這是代碼：

import requests
from bs4 import BeautifulSoup
import re

i = 0
for i in range(0, 12):
    i = i + 1
    request_result=requests.get("https://www.futwiz.com/en/fifa21/players?page=" + str(i) + "&release=icons")
    src = request_result.content
    soup = BeautifulSoup(src, features="html.parser")


    results = soup.find("table", class_="table results playersearchresults")

    # Gets all the names and stores them in a file
    names = results.select('a b')
    with open("names.txt", "w+") as f:
    
        for name in names:
            data = name.get_text()
            f.write("\n")
            f.write(data)            

    # Gets all the OVR values and stores them in a file
    OVR_values = results.select('div .otherversion21-txt')
    with open("OVR.txt", "w+") as f:
        for OVR_value in OVR_values:
            data = OVR_value.get_text()
            f.write("\n")
            f.write(data)
            

提前致謝！

Answer 1

在您的代碼中嘗試a+而不是w+ 。 如果您使用w+ ，則現有內容將被新內容覆蓋。 如果您使用a+ ，則新數據將附加到文件中的現有內容中。

import requests
from bs4 import BeautifulSoup
import re

i = 0
for i in range(0, 12):
    i = i + 1
    request_result=requests.get("https://www.futwiz.com/en/fifa21/players?page=" + str(i) + "&release=icons")
    src = request_result.content
    soup = BeautifulSoup(src, features="html.parser")


    results = soup.find("table", class_="table results playersearchresults")

    # Gets all the names and stores them in a file
    names = results.select('a b')
    with open("names.txt", "a+") as f:
    
        for name in names:
            data = name.get_text()
            f.write("\n")
            f.write(data)            

    # Gets all the OVR values and stores them in a file
    OVR_values = results.select('div .otherversion21-txt')
    with open("OVR.txt", "a+") as f:
        for OVR_value in OVR_values:
            data = OVR_value.get_text()
            f.write("\n")
            f.write(data)