[英]How to iterate mongodb database in node.js to send to Algolia?
In the documentation of Algolia, for the node.js part they specified to use MySQL for indexing but not MongoDB, I have another question regarding this issue but it is more a general question , check here 在Algolia的文档中,对于node.js部分,他们指定使用MySQL进行索引但不使用MongoDB,我还有另外一个关于这个问题的问题,但这是一个更普遍的问题, 请点击这里
Some folks ask me to use mongo-connector but tried it and I got some unknown error, which got me to square one 有些人要求我使用mongo-connector,但尝试了它,我得到了一些未知的错误,这让我变得正方形
My real question is, how do i iterate a list of collections in mongodb to algolia? 我真正的问题是,如何将mongodb中的集合列表迭代到algolia?
This is the Algolia's version of MySQL in Node.js 这是Algolia在Node.js中的MySQL版本
var _ = require('lodash');
var async = require('async');
var mysql = require('mysql');
var algoliasearch = require('algoliasearch');
var client = algoliasearch("RQGLD4LOQI", "••••••••••••••••••••••••••••••••");
var index = client.initIndex('YourIndexName');
var connection = mysql.createConnection({
host: 'localhost',
user: 'mysql_user',
password: 'mysql_password',
database: 'YourDatabaseName'
});
connection.query('SELECT * FROM TABLE_TO_IMPORT', function(err, results, fields) {
if (err) {
throw err;
}
// let's use table IDS as Algolia objectIDs
results = results.map(function(result) {
result.objectID = result.id;
return result;
});
// split our results into chunks of 5,000 objects, to get a good indexing/insert performance
var chunkedResults = _.chunk(results, 5000);
// for each chunk of 5,000 objects, save to algolia, in parallel. Call end() when finished
// or if any save produces an error
// https://github.com/caolan/async#eacharr-iterator-callback
async.each(chunkedResults, index.saveObjects.bind(index), end);
});
function end(err) {
if (err) {
throw err;
}
console.log('MySQL<>Algolia import done')
};
To be specific I'm using mongoose as my ORM, so I have no experience in other libraries. 具体来说我使用mongoose作为我的ORM,所以我没有其他库的经验。 Please help me on this, so that I could some searching interface already :(.
请帮我这个,以便我可以找到一些搜索界面:(。
You can use the following code to iterate over the whole MongoDB mydb.myCollection
collection + create batches that will be sent to the Algolia index: 您可以使用以下代码迭代整个MongoDB
mydb.myCollection
集合+创建将发送到Algolia索引的批处理:
var Db = require('mongodb').Db,
Server = require('mongodb').Server,
algoliasearch = require('algoliasearch');
// init Algolia index
var client = algoliasearch("*********", "••••••••••••••••••••••••••••••••");
var index = client.initIndex('YourIndexName');
// init connection to MongoDB
var db = new Db('mydb', new Server('localhost', 27017));
db.open(function(err, db) {
// get the collection
db.collection('myCollection', function(err, collection) {
// iterate over the whole collection using a cursor
var batch = [];
collection.find().forEach(function(doc) {
batch.push(doc);
if (batch.length > 10000) {
// send documents by batch of 10000 to Algolia
index.addObjects(batch);
batch = [];
}
});
// last batch
if (batch.length > 0) {
index.addObjects(batch);
}
});
});
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.