[英]Import sql file in node.js and execute against PostgreSQL
I'm looking for an efficient way to take a raw sql file and have it executed synchronously against a postgres database, akin to if you ran it through psql
.我正在寻找一种有效的方法来获取原始 sql 文件并让它针对 postgres 数据库同步执行,类似于通过
psql
运行它。
I have an sql file which creates all databases, imports data, etc. I need to execute this using node.js but cannot find any module which does this automatically.我有一个 sql 文件,它创建所有数据库、导入数据等。我需要使用 node.js 执行它,但找不到任何自动执行此操作的模块。 For the node.js application itself, we use node-postgres ('pg'), knex.js and bookshelf.js.
对于 node.js 应用程序本身,我们使用 node-postgres ('pg')、knex.js 和 bookshelf.js。 I assume though that pg is best for this.
我认为 pg 最适合这个。
One alternative I can think of is to read the full file, split it by semicolons, replace newlines with spaces, trim any duplicate space, then feed it into pg one by one in a manner that they're executed sequentially, not asynchronously.我能想到的一种替代方法是读取整个文件,用分号分割,用空格替换换行符,修剪任何重复的空格,然后以顺序执行而不是异步执行的方式将其一个一个地输入 pg。 I'm a little surprised if this is truly the most efficient way and also if no libraries exist yet to solve this.
如果这真的是最有效的方法,并且还没有库来解决这个问题,我有点惊讶。 I'm a little hesitant to jump into it seeing as SQL syntax can itself be a little challenging and I might accidentally mash it up.
我有点犹豫要不要跳进去,因为 SQL 语法本身就有点挑战性,我可能会不小心把它混搭。
Some clarifications in advance:提前说明几点:
psql
cannot be used as it's not installed on the target machine psql
不能使用,因为它没有安装在目标机器上You can just separate consequent queries with a semicolon when passed to client.query
传递给
client.query
时,您可以使用分号分隔后续查询
That works:那个有效:
var pg = require('pg');
pg.connect('postgres://test:test@localhost/test', function(err, client, done){
client.query('CREATE TABLE test (test VARCHAR(255)); INSERT INTO test VALUES(\'test\') ');
done();
});
And consequently, that works too:因此,这也有效:
var pg = require('pg');
var fs = require('fs');
var sql = fs.readFileSync('init_database.sql').toString();
pg.connect('postgres://test:test@localhost/test', function(err, client, done){
if(err){
console.log('error: ', err);
process.exit(1);
}
client.query(sql, function(err, result){
done();
if(err){
console.log('error: ', err);
process.exit(1);
}
process.exit(0);
});
});
I've written the following function which works for my case.我编写了以下适用于我的案例的函数。 It would have been much more simpler if it weren't for:
如果不是因为:
batch
to manage concurrencybatch
来管理并发Code snippet:代码片段:
function processSQLFile(fileName) {
// Extract SQL queries from files. Assumes no ';' in the fileNames
var queries = fs.readFileSync(fileName).toString()
.replace(/(\r\n|\n|\r)/gm," ") // remove newlines
.replace(/\s+/g, ' ') // excess white space
.split(";") // split into all statements
.map(Function.prototype.call, String.prototype.trim)
.filter(function(el) {return el.length != 0}); // remove any empty ones
// Execute each SQL query sequentially
queries.forEach(function(query) {
batch.push(function(done) {
if (query.indexOf("COPY") === 0) { // COPY - needs special treatment
var regexp = /COPY\ (.*)\ FROM\ (.*)\ DELIMITERS/gmi;
var matches = regexp.exec(query);
var table = matches[1];
var fileName = matches[2];
var copyString = "COPY " + table + " FROM STDIN DELIMITERS ',' CSV HEADER";
var stream = client.copyFrom(copyString);
stream.on('close', function () {
done();
});
var csvFile = __dirname + '/' + fileName;
var str = fs.readFileSync(csvFile);
stream.write(str);
stream.end();
} else { // Other queries don't need special treatment
client.query(query, function(result) {
done();
});
}
});
});
}
Beware that this would fail if you used semicolons anywhere except to terminate SQL statements.请注意,如果您在除终止 SQL 语句之外的任何地方使用分号,这都会失败。
The @databases/pg
client supports running SQL files out of the box:@databases/pg
客户端支持开箱即用的运行 SQL 文件:
const createPool = require('@databases/pg');
const {sql} = require('@databases/pg');
const db = createPool();
db.query(sql.file('my-file.sql')).catch(ex => {
console.error(ex);
process.exitCode = 1;
}).then(() => db.dispose());
It also supports having multiple statements in a single call to db.query
:它还支持在一次调用
db.query
中包含多个语句:
const createPool = require('@databases/pg');
const {sql} = require('@databases/pg');
const db = createPool();
db.query(sql`
INSERT INTO users (name) VALUES (${'Forbes'});
SELECT * FROM users;
`)).then(
results => console.log(results)
).catch(ex => {
console.error(ex);
process.exitCode = 1;
}).then(() => db.dispose());
In this example, each statement is run in sequence, and the result of the last statement is returned.本例中,每条语句依次运行,返回最后一条语句的结果。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.