I am trying to scrape a classified ad search result page.
I have tried console logging everything I can to make sure I am getting a response, which I am, but when I actually use cheerio to query something I don't get anything back. For instance if I just query for how many children using $('.listing-group').children('section').length
I get back 0
instead of 24
when I console log it.
Here is what I'm doing. Pardon the long URL.
const request = require("request");
const cheerio = require("cheerio");
const app = express();
app.get("/scrape", function(req, res) {
url =
"http://classifieds.ksl.com/search/?keyword=code&category%5B%5D=Books+and+Media&zip=&miles=25&priceFrom=&priceTo=&city=&state=&sort=";
request(url, function(error, response, html) {
if (!error) {
let $ = cheerio.load(html);
let test = $("#search-results").find("section").length
console.log(test);
} else {
console.log("there has been an error");
}
res.send("Check the console.");
});
});
app.listen("8081");
console.log("check localhost:8081/scrape");
exports = module.exports = app;
I'm new to cheerio so I'm assuming I'm probably making a simple error, but with all the tutorials I've checked, I can't seem to find an answer.
Ultimately I want to grab each result on the page (found in the 'section' tags) to display the data for a personal project I'm working on.
看起来像:
JSON.parse(html.match(/listings: (\[.*\])/)[1])
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.