How to read multiple csv files using node.js from a directory?

Question

I want to read multiple CSV files from a directory and store them in a database. I tried a lot and also found this code to read files. Apparently, it just scans the names and does not return the actual files. Could anyone please tweak something in this code and give me a solution?

const fs = require('fs');
//joining path of directory 
const directoryPath = path.join(__dirname, 'Documents');
//passsing directoryPath and callback function
fs.readdir(directoryPath, function (err, files) {
    //handling error
    if (err) {
        return console.log('Unable to scan directory: ' + err);
    } 
    //listing all files using forEach
    files.forEach(function (file) {
        // Do whatever you want to do with the file
        console.log(file); 
    });
});```

Answer 1

So, I looked around the internet for days but didn't get a satisfactory answer for this. So, thought I'd mix the reading of a single file concept and then club it to the multiple files. Here is how I did it:

Let's suppose we have a directory called csvFiles and we want to read all the CSV files present inside it.

First of all, install and require all the dependencies.

var fs = require('fs');
const path = require('path');
const mysqlConnection = require('./connection');
const fastcsv = require('fast-csv');

Then make a function that returns a Promise with the filenames.

//Hard coded directory has been used.
//Put your path here...
const currDir = path.join(__dirname + '/../csvfiles/');

// Function to get the filenames present
// in the directory
const readdir = (dirname) => {
  return new Promise((resolve, reject) => {
    fs.readdir(dirname, (error, filenames) => {
      if (error) {
        reject(error);
      } else {
        resolve(filenames);
      }
    });
  });
};

Make a CSV filter to filter out the CSV files present inside the directory.

//CSV filter to filter out the csv files
//in the directory
const filtercsvFiles = (filename) => {
  return filename.split('.')[1] == 'csv';
};

Then read all the files and store them in an array. Start parsing all the files inside a for loop and save it in mysql.

readdir(currDir).then((filenames) => {
  filenames = filenames.filter(filtercsvFiles);

  for (let i = 0; i < filenames.length; i++) {
    let currFilePath = currDir + filenames[i];

    //Use fast-csv to parse the files
    let csvData = [];
    fastcsv
      .parseFile(currFilePath)
      .on('data', (data) => {
        csvData.push(data);
      })
      .on('end', () => {
        csvData.shift();

        //Save the data in mysql
        query =
          'insert ignore into csvData (id, name, date, steps, calories) values ?';
        mysqlConnection.query(query, [csvData], (err, response) => {
          console.log(err || response);
        });
      });
  }
});

And that's how you read any number of CSV files present inside of a directory. Your welcome:)