简体   繁体   English

HTTP 客户端无法获取页面,而其他工具(wget/curl)可以

[英]HTTP client can't fetch a page while other tools (wget/curl) can

The following code in Go can't fetch page, it gets 404 error, while code in another language (php+curl) has no issues with this page. Go 中的以下代码无法获取页面,它得到 404 错误,而另一种语言(php+curl)的代码与该页面没有问题。 What is a reason of such behaviour?这种行为的原因是什么?

package main

import (
     "fmt"
    "net/http"
)

func main() {
    client := http.Client {}
    req, err  := http.NewRequest("GET", "https://myhresschoolofmusic.com/", nil)

    resp, err := client.Do(req)
    if err != nil {
    fmt.Println(err)
    panic(err)
    }
    defer resp.Body.Close()

    fmt.Printf("%#v\n", resp)
}

Output of this code此代码的 Output

&http.Response{
Status:"404 Not Found", 
StatusCode:404, 
Proto:"HTTP/2.0",
 ProtoMajor:2, 
ProtoMinor:0, 
Header:http.Header{"Content-Length":[]string{"1245"}, 
"Content-Type":[]string{"text/html"}, 
"Date":[]string{"Wed, 20 Jan 2021 06:44:22 GMT"}, 
"X-Powered-By":[]string{"ASP.NET"}}, 
....

Seems that page is very picky about User-Agent header and rejects Go net/http default value Go-http-client/1.1 .似乎该页面对User-Agent header 和拒绝 Go net/http默认值Go-http-client/1.1非常挑剔。 With browser-like user agent value like:使用类似浏览器的用户代理值,例如:

req, err := http.NewRequest("GET", "https://myhresschoolofmusic.com", nil)
req.Header.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36")

it works just fine.它工作得很好。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM