Scraping text data from different domain urls using Python

Question

Is there any way to scrape only the text data from different domain urls in Python?

For example in this website the text is in a different block than in this page. I would like to write a single function that would allow me to scrape the text from both these websites at the same time. Is that possible in Python?

Answer 1

The only possible thing in python is to scrape the whole text of a page. You can do that using that code.

import requests
from bs4 import BeautifulSoup
r = requests.get('https://www.businessinsider.in/tech/news/airbnb-is-getting-ripped-apart-for-asking-renters-to-donate-money-to-landlords/articleshow/76968577.cms')
soup = BeautifulSoup(r.text, 'html.parser')
texet = soup.find('html').text
print(texet)