简体   繁体   English

通过file_get_contents发送url参数返回nothig

[英]Sending url parameters through file_get_contents returns nothig

I am trying to scrape a website in order to get latitude and longitude for counties in the us(there are 3306 thus why I am trying to do it through code and not manually) 我正在尝试抓取一个网站,以获取美国各县的经度和纬度(因此有3306个,为什么我试图通过代码而不是手动进行操作)

I am using the code below 我正在使用下面的代码

function GetLatitude($countyName,$stateShortName){
        //Create DOM from url
        $page = file_get_contents("https://www.mapdevelopers.com/geocode_tool.php?$countyName,$stateShortName");
        $doc = new DOMDocument();
        $doc->loadHTML($page);
        $node = $doc->getElementById("display_lat");

        var_dump($doc);
    }

    GetLatitude("Guilford County","NC");

This returns nothing but if I change the url to get without the parameters like " https://www.mapdevelopers.com/geocode_tool.php " then I can see that $doc now has some information in it but that is not useful because the value I need (latitude) is dependent upon the parameters passed into the url. 这不会返回任何内容,但是如果我更改url以不使用“ https://www.mapdevelopers.com/geocode_tool.php ”之类的参数,则可以看到$doc现在包含一些信息,但这没有用,因为我需要的值(纬度)取决于传递到url中的参数。

How do I solve this issue? 我该如何解决这个问题?

EDIT: 编辑:

Based on the suggestion to encode the parameters I changed my code to this and now the document contains information but appears as though it is ignoring the parameters 基于对参数进行编码的建议,我将代码更改为此,现在文档包含信息,但看起来好像是在忽略参数

<?
function GetLatitude($countyName,$stateShortName){
    $countyName = urlencode($countyName);
    $stateShortName = urlencode($stateShortName);
    //Create DOM from url
    $page = file_get_contents("https://www.mapdevelopers.com/geocode_tool.php?address=$countyName,$stateShortName");
    $doc = new DOMDocument();
    $doc->loadHTML($page);
    $node = $doc->getElementById("display_lat");

    var_dump($doc);
    }

    GetLatitude("Clarke County","AL");
?>

Your issue is that the latitude information etc isn't present on page load, and java script puts it there 您的问题是页面加载中不存在纬度信息等,而Java脚本将其放置在那里

You're going to have a hard time trying to run a webpage with JS and scraping it from PHP without something in the middle, maybe re-try this project with something like puppet or phantomjs so you can run your script against a real browser. 您将很难尝试使用JS运行网页并从PHP抓取网页而中间没有任何东西,也许要用puppet或phantomjs之类的东西重试此项目,以便可以在真正的浏览器上运行脚本。

搜索页面上有一个对https://www.mapdevelopers.com/data.php的ajax请求。发送POST或GET请求将为您提供所需的响应

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM