如何提高 JS 中函数式代码的性能

Question

I am trying to learn functional programming and I cannot figure this one out. In my minimal working example, I have a list of dictionaries, each dictionary containing a filename and the size of the file. I want to create a new dictionary that groups the files by size (this is part of a larger algorithm that finds duplicated files).

Here is the "traditional" approach to this, mutating data:

const groupFilesBySize = (allFileData) => {
  const filesSortedBySize = {};
  for (const fileData of allFileData) {
    if (fileData.size in filesSortedBySize) {
      filesSortedBySize[fileData.size].push(fileData.file);
    } else {
      filesSortedBySize[fileData.size] = [fileData.file];
    }
  }

  return filesSortedBySize;
};

And here is my "best" attempt to do it in a functional way:

const groupFilesBySizeFunctional = (allFileData) =>
  allFileData.reduce(
    (filesSortedBySize, fileData) => ({
      ...filesSortedBySize,
      [fileData.size]: filesSortedBySize[fileData.size]
        ? [...filesSortedBySize[fileData.size], fileData.file]
        : [fileData.file]
    }),
    {}
  );

I have benchmarked them (reproducible example below) and the functional version is about 10000 times slower . This is no joke --it is just plain unusable. I can imagine that creating a new dictionary every time we process a file in reduce is what is causing the delay.

Nonetheless, now I see two possibilities: either functional programming has a terrible performance, or I cannot write proper functional code. For obviously the second one is right, I would like to ask: what is the proper way of writing the function groupFilesBySize in a functional way?

---

Benchmark: Use this function to obtain the array of files paths and file sizes:

async function walk(dir) {
  let files = [];
  files = await fs.readdir(dir);
  const parsedFiles = await Promise.all(files.map(async (fileName) => {
    const filePath = path.join(dir, fileName);

    const stats = await fs.lstat(filePath);
    if (stats.isSymbolicLink() || stats.size === 0) {
      return null;
    }
    if (stats.isDirectory()) {
      return walk(filePath);
    } else if (stats.isFile()) {
      return { file: filePath, size: stats.size };
    }
  }));

  return parsedFiles.reduce(
    (all, folderContents) => (folderContents ? all.concat(folderContents) : all),
    []
  );
}

Then benchmark everything using:

const benchMark = async () => {
  const dir = path.dirname(__filename);
  const allFileData = await walk(dir);
  console.log(`Total files: ${allFileData.length}`);

  let start = new Date();
  const result1 = groupFilesBySize(allFileData);
  const time1 = new Date() - start;

  start = new Date();
  const result2 = groupFilesBySizeFunctional(allFileData);
  const time2 = new Date() - start;

  console.log('\nFINAL REPORT:')
  console.log(`Are results equal? ${JSON.stringify(result1) === JSON.stringify(result2)}`);
  console.log(`Non functional approach: ${time1} ms`);
  console.log(`Functional approach: ${time2} ms`);
};

To have sizable data, I chose to install the node package eslint , so that I have to group all files in the node_modules folder: npm install eslint . Output in my machine:

Total files: 6229

FINAL REPORT:
Are results equal? true
Non functional approach: 6 ms
Functional approach: 34557 ms

Answer 1

If you want to use make use of the functional programming paradigm then make sure that you're using functional data structures, such as those provided by Immutable.js .

 const { Map, List } = Immutable; const groupFilesBySize = allFileData => allFileData.reduce((filesSortedBySize, { size, file }) => filesSortedBySize.update(size, List(), list => list.push(file)), Map()); const allFileData = [ { size: 12, file: "Hello World!" }, { size: 3, file: "foo" }, { size: 3, file: "bar" }, { size: 6, file: "foobar" }, { size: 12, file: "Hello World!" }, { size: 4, file: "fizz" }, { size: 4, file: "buzz" }, { size: 8, file: "fizzbuzz" }, ]; console.time("groupFilesBySize"); for (let i = 0; i < 1e6; i++) groupFilesBySize(allFileData); console.timeEnd("groupFilesBySize"); console.log(groupFilesBySize(allFileData));

 <script src="https://cdnjs.cloudflare.com/ajax/libs/immutable/4.0.0-rc.12/immutable.min.js"></script>

On my machine it takes about 3 seconds to run one million iterations. Compare that to your original solution.

 const groupFilesBySize = (allFileData) => { const filesSortedBySize = {}; for (const fileData of allFileData) { if (fileData.size in filesSortedBySize) { filesSortedBySize[fileData.size].push(fileData.file); } else { filesSortedBySize[fileData.size] = [fileData.file]; } } return filesSortedBySize; }; const allFileData = [ { size: 12, file: "Hello World!" }, { size: 3, file: "foo" }, { size: 3, file: "bar" }, { size: 6, file: "foobar" }, { size: 12, file: "Hello World!" }, { size: 4, file: "fizz" }, { size: 4, file: "buzz" }, { size: 8, file: "fizzbuzz" }, ]; console.time("groupFilesBySize"); for (let i = 0; i < 1e6; i++) groupFilesBySize(allFileData); console.timeEnd("groupFilesBySize"); console.log(groupFilesBySize(allFileData));

On my machine it takes about 400 milliseconds to run one million iterations. Hence, the functional program is only about 10x slower than the imperative program.

In conclusion, don't use the functional programming paradigm with imperative data structures like objects and arrays. It's slow and it's messy. Use functional data structures instead.

Answer 2

If you mutate inside the reduce there is no problem, and you will improve a little performance.

const groupFilesBySizeFunctional = allFileData =>
  allFileData.reduce(
    (filesSortedBySize, fileData) =>
      Object.assign(filesSortedBySize, {
        [fileData.size]: filesSortedBySize[fileData.size]
          ? [...filesSortedBySize[fileData.size], fileData.file]
          : [fileData.file]
      }),
    {}
  );