python flask 请求在本地主机上有效，但在远程服务器上无效

Question

I'm trying to make a wallpaper page from the website "https://www.wallpaperflare.com",

When I try to run it on localhost it always works and displays the original page of the website.

But when I deploy to Heroku the page doesn't display the original page from the website, but "Error Get Request, Code 403" Which means requests don't work on that url.

This is my code:

@app.route("/wallpapers", methods=["GET"])
def wallpaper ():
    page = requests.get("https://www.wallpaperflare.com")
    if page.status_code == 200:
        return page.text
    else:
        return "Error Get Request, Code {}".format(page.status_code)

is there a way to solve it?

Answer 1

HTTP Error code 403 means Forbidden . You can read more here

It means wallpaperflare.com is not allowing you to make the request. This is because websites generally do not want scripts to be making requests to them. Make sure to read robots.txt of a site to see it's script crawling policies. More on that here

It works on your local machine as it is not yet blacklisted by wallpaperflare.com

Answer 2

Two things here:

• the user agent - unless you spoof it, the request module is going to use its own string and it's very obvious you are a bot
• the IP address - your server IP address may be denied for various reasons, whereas your home IP address works just fine.

It is also possible that the remote site applies different policies based on the client, if you are a bot then you might be allowed to crawl a bit but rate limiting measures could apply for example.