简体   繁体   中英

YQL query redirected from target url

I'm trying to scrap a website but when I try to connect to it using YQL, I get redirected to the homepage of the website instead of the page I'm trying to get content off.

Do anybody know what I could do to prevent my request being redirected or any solution to avoid this issue ?

Here is a like to the request I'm trying to perform and which is failing : Target site :
http://gticket.imagix.be/os1.aspx
Request in Yahoo Console :
http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Fgticket.imagix.be%2Fos1.aspx%22&diagnostics=true

It's not because of yql , actually it has 302 redirect. If you directly put this url in the browser's address bar or click it, then you can see that it has been redirected to the home page of the site and you can't prevent it.

This is the yql result of the page after redirection.

Update:

Also remember that if a website chooses to block YQL using the robots.txt directive, you won't be allowed to access it. So a site can reject yql request if it has been setup in that way and here is an article about blocking yql .

There is a followRedirects option in YQL which you can use. Check here

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM