NodeJS +请求 - 请求网站时拒绝访问
[英]NodeJS + Request - Access denied when requesting website
问题描述
I'm trying to request the html of a website using request but I keep getting an access denied error. 
我正在尝试使用请求请求网站的html，但我一直收到访问被拒绝错误。 How do I get past this? 我如何通过这个？ Here is the code for the function below: 以下是该函数的代码：
const request = require('request');
function firstShoe() {
request('https://www.jdsports.co.uk/product/green-nike-vapormax/281735/', function (error, response, body) {
console.log('body:', body);
});
}
Error: 错误：
</BODY>
</HTML>
body: <HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>
You don't have permission to access "http://www.jdsports.co.uk/product/green-nike-vapormax/281735/" on this server.<P>
Reference #18.609d3e17.1500116386.15f0cb85
</BODY>
</HTML>
Found a solution by passing the user-agent into the headers. 通过将用户代理传递到标头中找到解决方案。
function firstShoe() {
var options = {
headers: {'user-agent': 'node.js'}
}
request('https://www.jdsports.co.uk/product/green-nike-vapormax/281735/', options, function (error, response, body) {
console.log(body);
message.channel.send(body);
});
}
1 个解决方案
解决方案1
4 已采纳 2017-07-15 14:17:49
You are getting a 403 Forbidden because that website is blocking all requests sent using non common user agents (basically they check User-Agent header). 您正在获得403 Forbidden因为该网站阻止了使用非常见用户代理发送的所有请求（基本上他们检查User-Agent标头）。 It is a very simple protection to avoid scrappers. 这是一个非常简单的保护措施，以避免刮板。
For example, if you send the following cURL using its standard User-Agent, the response is received perfectly: 例如，如果您使用其标准User-Agent发送以下cURL，则会完美地收到响应：
curl -v 'https://www.jdsports.co.uk/product/green-nike-vapormax/281735/'
Nevertheless, if you repeat that request specifying a non existing User-Agent, the request is blocked: 但是，如果您重复指定不存在的User-Agent的请求，则会阻止该请求：
curl -v 'https://www.jdsports.co.uk/product/green-nike-vapormax/281735/' -H 'User-Agent: StackOverflow'
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.