[英]NodeJS crawler undefined is not a function
I'm using crawler for NodeJs 我正在对NodeJ使用搜寻器
this is my code: 这是我的代码:
var Crawler = require("crawler");
//var jsdom = require('jsdom');
var url = require('url');
var fs = require('fs');
if (typeof String.prototype.startsWith != 'function') {
// see below for better implementation!
console.log("added");
String.prototype.startsWith = function (str){
return this.indexOf(str) == 0;
};
}
var c = new Crawler({
maxConnections: 10,
// This will be called for each crawled page
callback: function (error, result,$) {
// $ is Cheerio by default
//a lean implementation of core jQuery designed specifically for the server
if(result.request.uri.href.startsWith("http://www.geocaching.com/geocache/")){
var titel = $('#ctl00_ContentBody_CacheName');
var coords = $('#uxLatLon');
console.log(titel +": "+ coords);
}
$('a').each(function(index, a) {
var toQueueUrl = $(a).attr('href');
c.queue(toQueueUrl);
});
}
});
c.queue('http://www.geocaching.com/');
but after it runs for a while, I get this error: 但运行了一段时间后,出现此错误:
TypeError: undefined is not a function
at Object.Crawler.callback (C:\Users\Lukas\Documents\Geocachcrawler\app.js:27:9)
at Crawler._onContent (C:\Users\Lukas\Documents\Geocachcrawler\node_modules\crawler\lib\crawler.js:462:17)
at Request._callback (C:\Users\Lukas\Documents\Geocachcrawler\node_modules\crawler\lib\crawler.js:352:18)
at Request.self.callback (C:\Users\Lukas\Documents\Geocachcrawler\node_modules\crawler\node_modules\request\request.js:236:22)
at Request.emit (events.js:98:17)
at Request.<anonymous> (C:\Users\Lukas\Documents\Geocachcrawler\node_modules\crawler\node_modules\request\request.js:1142:14)
at Request.emit (events.js:117:20)
at IncomingMessage.<anonymous> (C:\Users\Lukas\Documents\Geocachcrawler\node_modules\crawler\node_modules\request\request.js:1096:12)
at IncomingMessage.emit (events.js:117:20)
at _stream_readable.js:943:16
The problem is on this line: 问题在这条线上:
$('a').each(function(index, a) {
You are reaching a page with no a
tags, so the jQuery object is empty, and can't have functions run on it. 您正在接近,没有一个页面
a
标签,所以jQuery对象是空的,且不能有它运行的功能。 You have to check to make sure it isn't empty before running the each function. 在运行每个函数之前,必须检查以确保其不为空。
var a = $('a');
if(a.length != 0){
$('a').each(function(index, a) {
var toQueueUrl = $(a).attr('href');
c.queue(toQueueUrl);
});
}
Update: I may be incorrect, JSfiddle doesn't throw this error with jQuery 1.11.0 . 更新:我可能是不正确的, JSfiddle不会在jQuery 1.11.0中引发此错误 。 What version of jQuery are you using?
您正在使用哪个版本的jQuery?
Edit: Are you sure jQuery is being included? 编辑:确定要包括jQuery吗? It may be throwing an error on the selector.
它可能在选择器上引发错误。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.