Web 使用 requests 模块抓取卡住而没有切割 kernel python

Question

I am web scraping content from a list of urls and printing the text in python using the bs4 and requests module. The problem is that the scraping always gets stuck on a random url without cutting.

Furthermore when I manually cut with ctrl c it doesn't properly cut as I can't seem to run any other code, as if something is going on in the background.

Before I scrape I check the response code is 200 and the code looks as such (this is the url it seems to get stuck on this time):

url = 'https://www.businessinsider.in/business/ecommerce/news/amazon-is-eyeing-india-startups-as-it-gears-up-for-a-fight-with-asia-richest-man-in-retail/articleshow/81773692.cms?utm_campaign=cityfalcon&utm_medium=cityfalcon&utm_source=cityfalcon'

response = requests.get(url)
if str(response) == '<Response [200]>':
    report = BeautifulSoup(response.content, 'lxml').text
print(report)

Does the requests module have a certain amount of times you can use it within an hour? Wouldn't anyone know how I could start to go about debugging a problem like this without any error?

Any further clarifications needed or code needed please let me know.

EDIT

This has happened again with a different URL. The response code was <Response [403]> for url= https://www.investing.com/news/stephens-stick-to-their-buy-rating-for-tyson-foods-2470535?utm_campaign=cityfalcon&utm_medium=cityfalcon&utm_source=cityfalcon

It also has got stuck on this one:

<Response [200]> https://www.benzinga.com/analyst-ratings/analyst-color/21/04/20568999/analysts-upgraded-amc-snap-united-airlines-and-tesla-in-the-past-week'

Once again it won't let me interrupt and continue working. For the 403 error, it should just bypass the condition anyway so I really do not understand as there is no error it just keeps running?

Answer 1

I run it about 100 times, without any exceptions.

but one note, it's not the pythonic way to check response status code like this

if str(response) == '<Response [200]>':

Use this code instead.

if response.status_code == 200:
    # Do staff