How can I make Requests wait for a 302 redirection to finish in Python?

Question

I am new to web scraping, and I'm trying to get some data from a site, but there is a waiting queue when you enter it (maybe ~15 seconds when I try in a real browser), which is another url.

When using Requests in Python, I am never getting redirected to the real url, even if it is a 302 redirect (which should be temporary).

Using this code:

s = requests.Session()
r = s.get(store, allow_redirects=True, headers=headers)
print(r.status_code)
print(r.history)

I get:

200
[<Response [302]>]

If I put allow_redirects=False , I just get an empty page. How can I wait for the waiting queue to finish, and redirect me to the real site, before using bs4 to fetch the data I want?

I tried to insert that in a loop to check for r.url every x seconds but it never changes. Also tried to get every x seconds but same result.

Answer 1

it seems, the response contains a javascript that do the redirect thing... so may be you need read the redirect url from that r.text,