Newspaper3k：有什么方法可以将多篇 web 文章下载到一个变量中？

Question

我正在尝试下载一些 web 文章进行解析。 它们是类似的文章（年度报告），为了简单起见，我希望将所有三篇文章下载到一个单一的输出/变量中。

当我分隔多个 url 时，代码可以工作，但是，只有第一个 url 被下载。

这是我尝试运行的代码示例：

from newspaper import Article

consolidated = Article('url1.com', 'url2.com', 'url3.com')
consolidate.download()
consolidated.parse()
consolidated.nlp()

result = consolidated.text
print(result)

url1

Answer 1

Article的构造函数定义如下：

class Article(object):
    """Article objects abstract an online news article page
    """
    def __init__(self, url, title='', source_url='', config=None, **kwargs):

因此，如果您执行Article('url1.com', 'url2.com', 'url3.com') ，则 Python 将其解释为

Article(url='url1.com', title='url2.com', source_url='url3.com')

这不是你想要的。

例如，根据您的使用情况，您可以将所有文章存储在字典中，如下所示：

from newspaper import Article
import nltk
nltk.download('punkt')

urls = ['https://edition.cnn.com/2021/03/24/americas/brazil-youth-covid-19-intl-latam/index.html', 
        'https://edition.cnn.com/2021/03/25/business/astrazeneca-covid-vaccine/index.html', 
        'https://edition.cnn.com/2021/03/25/india/india-covid-double-mutant-variant-intl-hnk/index.html']

results = {}

for url in urls:
    article = Article(url)
    article.download()
    article.parse()
    article.nlp()
    results[url] = article

然后，您可以稍后像这样使用它：

for url in urls:
    print(url)
    article = results[url]
    print(article.authors)
    print(article.publish_date)
    print(article.keywords)
    print(article.summary)

本例中的输出：

https://edition.cnn.com/2021/03/24/americas/brazil-youth-covid-19-intl-latam/index.html
['Matt Rivers']
2021-03-24 00:00:00
['doctors', 'sick', 'silva', 'younger', 'icu', 'p1', 'patients', 'variant', 'getting', 'young', 'brazil', 'da', 'covid19']
São Paulo (CNN) It took only 10 days from start to finish, from the time 28-year-old Graciane da Silva got sick to the time she died.
She was alone when she passed away in a Rio de Janeiro Hospital -- her mom, Maria da Penha da Silva Siqueira, thinks about that often.
And amid that surge, a worrying pattern has emerged—more young people seem to be getting severely ill and dying from Covid-19, doctors tell CNN.
The increase in both illness and death in younger people has coincided with the rise of at least one Covid-19 variant in Brazil.
It's an effect that Maria da Penha da Silva Siqueira feels acutely nearly every day.
https://edition.cnn.com/2021/03/25/business/astrazeneca-covid-vaccine/index.html
['Julia Horowitz', 'Cnn Business', 'Andrew Berens', 'Svb Leerink Analyst', 'Ronn Torossian', 'Ceo Of Public Relations']
2021-03-25 00:00:00
['think', 'astrazenecas', 'mistake', 'trials', 'vaccine', 'data', 'doses', 'pandemic', 'european', 'astrazeneca', 'hero', 'went', 'villain', 'vaccines', 'company']
AstraZeneca updated its data on Thursday, reporting that the trials showed its vaccine to be 76% effective in preventing Covid-19 symptoms.
The AstraZeneca vaccine is administered to a patient at a pharmacy in London.
France also initially limited AstraZeneca vaccines to those under 65.
A nurse prepares a dose of the AstraZeneca vaccine at the Edouard Herriot hospital on Feb. 6 in Lyon, France.
Frustrations boiled over this week after 29 million doses of AstraZeneca's vaccine were discovered in a reported "raid" on a factory in Italy.
https://edition.cnn.com/2021/03/25/india/india-covid-double-mutant-variant-intl-hnk/index.html
['Jessie Yeung', 'Esha Mitra']
2021-03-25 00:00:00
['india', 'mutations', 'strain', 'variants', 'according', 'double', 'cases', 'vaccine', 'spike', 'variant', 'mutant', 'ministry', 'covid19', 'detects']
New Delhi (CNN) India has discovered a new "double mutant" variant of Covid-19 , as the country struggles to contain a spike in cases that's raising fears of a second wave.
A "double mutant" variant is a virus strain that carries two mutations.
India has now reported a total of more than 11.7 million cases and 160,000 related deaths, according to Johns Hopkins University data.
And now double mutations have been reported.
The Brazil variant known as P.1 has 17 mutations, and the South Africa variant known as B.1.351 also has multiple mutations, according to the US Centers for Disease Control and Prevention (CDC).