![](/img/trans.png)
[英]Inserting or Updating multiple values into a MySQL table using Node.js
[英]Node.js puppeteer mysql - Inserting fetched values in database inside a loop using mysql
我正在使用node.js和puppeteer來獲取一些數據。 ...現在我想使用mysql將獲取的數據插入數據庫中。 下面的方法似乎起作用了……但令我困惑的是,在console.log('DB插入成功。記錄:'+ i);中。 總是在后面,一段時間后它會停止...盡管仍然有帶有記錄的表。
那是我的應用程序:
let tableCell01;
let tableCell01Val;
let tableCell02;
let tableCell02Val;
const tableRows = await page.$$('table.tableFile2 > tbody > tr');
for (let i=1; i < tableRows.length; i++){
tableRow = tableRows[i];
tableCell01 = await tableRow.$('td:nth-child(1) a');
tableCell01Val = await page.evaluate( tableCell01 => tableCell01.innerText, tableCell01 );
tableCell02 = await tableRow.$('td:nth-child(2)');
tableCell02Val = await page.evaluate( tableCell02 => tableCell02.innerText, tableCell02 );
tableCell02ValA.replace(/(^\s+|\s+$)/g,'');
console.log('\n');
console.log('ID: '+tableCell01Val);
console.log('Company: '+tableCell02Val);
console.log('Iterator: '+i);
const insertCompanyList = "INSERT INTO companyList ( company_name, id ) values (?,?)";
connection.query(insertCompanyList,[tableCell02Val, tableCell01Val],function(err, rows) {
if (err) {
console.log(err);
} else {
console.log('DB insert successful. Record: '+i);
}
});
}
我可以在控制台中看到:
ID: 3136
Company: Company A
Iterator: 1
ID: 3143
Company: Company B
Iterator: 2
DB insert successful. Record: 1
ID: 4497
Company: Company C
Iterator: 3
ID: 3164
Company: Company D
Iterator: 4
ID: 3219
Company: Company E
Iterator: 5
ID: 3071
Company: Company F
Iterator: 6
ID: 3184
Company: Company G
Iterator: 7
DB insert successful. Record: 2
ID: 3130
Company: Company H
Iterator: 8
DB insert successful. Record: 3
DB insert successful. Record: 4
DB insert successful. Record: 5
DB insert successful. Record: 6
DB insert successful. Record: 7
DB insert successful. Record: 8
ID: 1844
Company: Company I
Iterator: 1
ID: 3687
Company: Company J
Iterator: 2
ID: 4514
Company: ECompany K
Iterator: 3
ID: 3635
Company: Company L
Iterator: 4
ID: 3884
Company: Company M
Iterator: 5
ID: 3482
Company: Company N
Iterator: 6
DB insert successful. Record: 1
ID: 3482
Company: Company O
Iterator: 7
ID: 1827
Company: Company P
Iterator: 8
DB insert successful. Record: 2
ID: 1827
Company: Company Q
Iterator: 9
ID: 6465
Company: Company R
Iterator: 10
ID: 0731
Company: Company S
Country: B9
Iterator: 11
No pagination!
DB insert successful. Record: 3
DB insert successful. Record: 4
DB insert successful. Record: 5
DB insert successful. Record: 6
DB insert successful. Record: 7
DB insert successful. Record: 8
DB insert successful. Record: 9
DB insert successful. Record: 10
DB insert successful. Record: 11
我想念什么? 我想我需要將連接查詢放在async.function中?! 就像這里: 在數據庫中的循環中插入值(問題):插入相同的值-節點js / sql 。
只是承諾connection.query,以便您await
它。 您發布的其他問題的鏈接與您的問題非常相似。
這個問題是問了個遍,因為這是很難理解的,但基本上connection.query
立即運行,跳到下一行,然后一些點后(當數據庫響應和事件循環有時間來處理它)的function(err, rows) {}
部分運行。 因此,在您的某些pepeteer等待(或其他異步進程)之間,它正在處理function(err,rows){}
。
下util.promisify
建議:學習使用util.promisify
! ( https://nodejs.org/dist/latest-v8.x/docs/api/util.html#util_util_promisify_original )
let tableCell01;
let tableCell01Val;
let tableCell02;
let tableCell02Val;
const tableRows = await page.$$('table.tableFile2 > tbody > tr');
for (let i=1; i < tableRows.length; i++){
tableRow = tableRows[i];
tableCell01 = await tableRow.$('td:nth-child(1) a');
tableCell01Val = await page.evaluate( tableCell01 => tableCell01.innerText, tableCell01 );
tableCell02 = await tableRow.$('td:nth-child(2)');
tableCell02Val = await page.evaluate( tableCell02 => tableCell02.innerText, tableCell02 );
tableCell02ValA.replace(/(^\s+|\s+$)/g,'');
console.log('\n');
console.log('ID: '+tableCell01Val);
console.log('Company: '+tableCell02Val);
console.log('Iterator: '+i);
const insertCompanyList = "INSERT INTO companyList ( company_name, id ) values (?,?)";
let rows = await new Promise((resolve,reject)=>{
connection.query(insertCompanyList,[tableCell02Val, tableCell01Val],function(err, rows) {
if (err) {
console.log(err);
reject(err);
} else {
console.log('DB insert successful. Record: '+i);
resolve(rows);
}
});
});
}
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.