Web 抓取錯誤 selenium: name 'records' is not defined

Question

我得到這個結果：

line 72, in <module>
    writer.writerows(records)
NameError: name 'records' is not defined

這是我的代碼：

def main(search_term):
    records = []
    url= get_url(search_term)
    
    for page in range(1,21):
        driver.get(url.format(page))
        soup = BeautifulSoup(driver.page_source, 'html.parser')
        results = soup.find_all('div',{'data-component-type': 's-search-result'})
        
        for item in results:
            record = extract_record(item)
            if record: 
                records.append(record)
            
    driver.close()        


#save data to csv file 
with open ('results.csv', 'w', newline='', encoding= 'utf-8') as f:
    writer = csv.writer(f)
    writer.writerow(['Title','Current Price', 'Original Price', 'Rating', 'Reviewscount','Reviews', 'Url'])
    writer.writerows(records)
    
main('Bluetooth earphones')

Answer 1

使您的main() function 返回records列表，並在writer.writerows()方法中使用返回值：

def main(search_term):
    records = []
    url = get_url(search_term)
    
    for page in range(1,21):
        driver.get(url.format(page))
        soup = BeautifulSoup(driver.page_source, 'html.parser')
        results = soup.find_all('div', {'data-component-type': 's-search-result'})
        
        for item in results:
            record = extract_record(item)
            if record: 
                records.append(record)
            
    driver.close()   
    return records # return list
  
records = main('Bluetooth earphones') # Get returned list

#save data to csv file 
with open ('results.csv', 'w', newline='', encoding= 'utf-8') as f:
    writer = csv.writer(f)
    writer.writerow(['Title','Current Price', 'Original Price', 'Rating', 'Reviewscount','Reviews', 'Url'])
    writer.writerows(records) # Use returned list

Answer 2

records = []在main(search_term)方法中定義：

def main(search_term):
    records = []
    

在main()之外訪問records = [] ：

writer.writerows(records)

您需要在全局 scope 中定義/聲明如下：

records = []
    def main(search_term):
        url= get_url(search_term)
        .
        .
        for item in results:
            record = extract_record(item)
            if record: 
                records.append(record)
        
#save data to csv file 
with open ('results.csv', 'w', newline='', encoding= 'utf-8') as f:
        writer = csv.writer(f)
        writer.writerow(['Title','Current Price', 'Original Price', 'Rating', 'Reviewscount','Reviews', 'Url'])
        writer.writerows(records)