简体繁体 English

使用scrapy crawler进行虚假地理定位

[英]fake geolocation with scrapy crawler

原文 2012-11-08 22:16:03 0 1 python/ web-crawler/ scrapy

I am trying to scrape a website which serves different page depending upon the geolocation of the IP sending the request.我正在尝试根据发送请求的 IP 的地理位置抓取一个提供不同页面的网站。 I am using an amazon EC2 located in US(which means it serves up a page meant for US) but I want the page that will be served in India.我正在使用位于美国的亚马逊 EC2（这意味着它提供了一个面向美国的页面），但我希望该页面将在印度提供。 Does scrapy provide a way to work around this somehow? scrapy 是否提供了一种以某种方式解决此问题的方法？

1 个解决方案

If the site you are scraping does IP based detection, your only option is going to be to change your IP somehow.如果您正在抓取的站点进行基于 IP 的检测，您唯一的选择就是以某种方式更改您的 IP。 This means either using a different server (I don't believe EC2 operates in India) or proxying your server requests.这意味着要么使用不同的服务器（我不相信 EC2 在印度运营）或代理您的服务器请求。 Perhaps you can find an Indian proxy service?也许您可以找到印度代理服务？