使用 Python 从不同的域 url 抓取文本数据

Question

Is there any way to scrape only the text data from different domain urls in Python?

For example in this website the text is in a different block than in this page. I would like to write a single function that would allow me to scrape the text from both these websites at the same time. Is that possible in Python?

Answer 1

The only possible thing in python is to scrape the whole text of a page. You can do that using that code.

import requests
from bs4 import BeautifulSoup
r = requests.get('https://www.businessinsider.in/tech/news/airbnb-is-getting-ripped-apart-for-asking-renters-to-donate-money-to-landlords/articleshow/76968577.cms')
soup = BeautifulSoup(r.text, 'html.parser')
texet = soup.find('html').text
print(texet)