使用curl带来来自外部站点的搜索结果

Question

I have 2 sites, one main, one external. On the main site, I am using Lucene to search through it. The problem is, I am trying to also search through the external site.

The Form action for the external site:

<form action="https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT" method="post" name="search_tribute" >

I've tried to use curl, but it only brings up the search form without actually doing the search (the field is empty as well).

<?php
$ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, tname='hello');
$output = curl_exec($ch);
echo $output;
curl_close($ch);
?>

Any tips?

I don't have access to the form action since it's on an external site. All i have is a form that links to it when I submit it.

Answer 1

<?php
$ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, array("teamName" => "hello", "searchType" => "team"));
$output = curl_exec($ch);
echo $output;
curl_close($ch);
?>

Can you try this? I'm pretty sure it's supposed to be teamName instead of tName

Answer 2

Try just putting it into an array. as that will be the variable the $_POST checks on the other side

and just checked your link, its teamName for the field

$fields = array("teamName"=>"julia");

Then..

curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);

So your complete code is...

<?php
 $ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
 $fields = array("teamName"=>"julia");
 curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
 curl_setopt($ch, CURLOPT_POST, true);
 curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
 $output = curl_exec($ch);
 var_dump($output);
 curl_close($ch);
 ?>

Answer 3

Most search engine use GET and not POST .. you can try

// asumption
$_POST['search'] = "hello";

// Return goole Search Result
echo curlGoogle($_POST['search']);

function curlGoogle($keyword) {
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, 'http://www.google.com/search?hl=en&q=' . urlencode($keyword) . '&btnG=Google+Search&meta=');
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_FILETIME, true);
    $data = curl_exec($ch);
    curl_close($ch);
    return $data;
}

Or if you want post then

curl_setopt($ch, CURLOPT_POSTFIELDS, array("search"=>"hello"));

Answer 4

Your php code is not valid syntax, it does not compile.

So if this is really what you have, your problem is that your file generates a fatal error.

That being said, this question is hard to answer since we don't know the site you want to grab your search results from.

Try modifying your line like this:

curl_setopt($ch, CURLOPT_POSTFIELDS, "search=hello");

or alternatively

curl_setopt($ch, CURLOPT_POSTFIELDS, array("search" => "hello");

Maby it will work, however it may be that more post data is required or that the element name is not correct.

You have to look at the form or try making a request and look at it with chromes developer tools or firebug.

Also there are a number of ways for external sites to prevent what you are doing, altough evertything can be worked around somehow.

Assuming that is not the case, I hope i could help you.