它仅打印该站点的最后一篇文章。 需要全部打印

Question

below is my code. print(news_csv) works fine and prints all the article I want but news_csv.to_csv('bbb.csv') prints out only the last article.

import pandas as pd
import requests
from bs4 import BeautifulSoup

source = requests.get('https://www.vanglaini.org/').text
soup = BeautifulSoup(source, 'lxml')
for article in soup.find_all('article'):
    if article.a is None:
        continue
    headline = article.a.text
    summary=article.p.text
    link = "https://www.vanglaini.org" +article.a['href']
    #print(headline)
    #print(summary)
    #print(link)
    news_csv = pd.DataFrame({'Headline': [headline],
                             'Summary': [summary],
                             'Link': [link],
                             })
    print(news_csv)
    news_csv.to_csv('bbb.csv')

#print()

Only the last article is printed in CSV help.

Answer 1

You have defined the variable news_csv inside the for loop. That means it will be overwritten each time you iterate over articles. That's why only the last article will be present in the csv file. In fact, the file is constantly overwritten.

Instead, your content should be appended to the container object and then saved as csv only once the for loop has completed.

If you really want to use pandas DataFrame, you should follow the very last example provided in the documentation: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.append.html

Append all articles content in a list and then generate the DataFrame object using pd.concat().

Here is how I would write it:

import pandas as pd
import requests
from bs4 import BeautifulSoup

source = requests.get('https://www.vanglaini.org/').text
soup = BeautifulSoup(source, 'lxml')
articles = []
for article in soup.find_all('article'):
    if article.a is None:
        continue
    headline = article.a.text
    summary=article.p.text
    link = "https://www.vanglaini.org" +article.a['href']

    articles.append((headline, summary, link))
    print(f'Headline: {headline}\nSummary: {summary}\nLink: {link}')
    #print(news_csv)

news_dataframe = pd.concat([pd.DataFrame([article], columns='Headline Summary Link'.split()) for article in articles ], ignore_index=True)

news_dataframe.to_csv('bbb.csv')