Scrape html with js
Question
I'm trying to get the html of www.soccerway.com . In particular this:
that have the label-wrapper class I also tried with: select.nav-select but I can't get any content. What I did is:
1) Created a php filed called grabber.php , this file have this code:
<?php echo file_get_contents($_GET['url']); ?>
2) Created a index.html file with this content:
<!DOCTYPE html>
<html>
<head>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js"></script>
<meta charset=utf-8 />
<title>test</title>
</head>
<body>
<div id="response"></div>
</body>
<script>
$(function(){
var contentURI= 'http://soccerway.com';
$('#response').load('grabber.php?url='+ encodeURIComponent(contentURI) + ' #label-wrapper');
});
var LI = document.querySelectorAll(".list li");
var result = {};
for(var i=0; i<LI.length; i++){
var el = LI[i];
var elData = el.dataset.value;
if(elData) result[el.innerHTML] = elData; // Only if element has data-value attr
}
console.log( result );
</script>
</html>
in the div there is no content grabbed, I tested my js code for get all the link and working but I've inserted the html page manually .
1 answers
solution1
2 ACCPTED 2016-04-05 20:02:29
I see a couple issues here.
var contentURI= 'http:/soccerway.com #label-wrapper';
You're missing the second slash in http:// , and you're passing a URL with a space and an ID to file_get_contents . You'll want this instead:
var contentURI = 'http://soccerway.com/';
and then you'll need to parse out the item you're interested in from the resulting HTML.
The #label-wrapper needs to be in the jQuery load() call, not the file_get_contents , and the contentURI variable needs to be properly escaped with encodeURIComponent :
$('#response').load('grabber.php?url='+ encodeURIComponent(contentURI) + ' #label-wrapper');
Your code also contains a massive vulnerability that's potentially very dangerous, as it allows anyone to access grabber.php with a url value that's a file location on your server. This could compromise your database password or other sensitive data on the server.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.