无法在函数内部调用非内置函数

Question

I have issue in my codes - there's a part in my codes make much calculations (30~40 seconds)

I need to reduce this time as much as possible - so I have to make use of my all CPU threads and distribute this part to be done more fast

for a few days - I made much researches and googled enough about how to write a multi-threaded code in nodejs , and I passed on many opinions about how nodejs is single threaded or multi-threaded , and readed about some topics like clustering , child-processes , and workers ..

I prefered to use a ready library to not dig into these topics too much , so I tried this library : https://github.com/Microsoft/napajs , but it fired some issues with me.

what I want should be simple - I just have array (ready for parallel computation - each thread ready to work on each element) [data,data,data,data]

I found this library : https://www.npmjs.com/package/paralleljs a bit simple and has this method (map) which great for my case

let's move to my code

initializing

var p = new Parallel([54,25,66,23,14,27,15,18] , { maxWorkers : 8 });

3 functions (fun is the main function which call for exports.calc and calcc)

exports.calc = function(i,h)
{
    return Math.log2(i*h);
}

function calcc(i,h)
{
    return Math.log2(i*h);
}

function fun(h)
{
    var sum = 0;
   for (var i = 1 ; i < 55555 ; i++)
   {
       for (var l = 1 ; l < 55555 ; l++)
       {
           sum+=(exports.calc(i,l)); // in this case calc is not a function error
           sum+=(calcc(i,l)); // in this case calcc is not defined error
       }
   } 
   return sum;
}

starting the parallel calculations using map function - this function should divide the array and let each thread to work on each element separately

p.map(fun).then(data => {
        console.log(data);
    });

the error always occurs in fun - anytime trying to call non built-in functions - for example

Math.max() , String.includes() and more ,, all these built-in functions are not making any issues , but any non built-in functions like calcc and exports.calc making issues

per the library docs , it uses the child processes - if this can help ,

if you can't help with this issue - but have other way to make this code possible (with any other libraries) - please share it

Answer 1

This can be implemented with Node cluster :

var cluster = require('cluster');

function calcc(i,h) {
    return Math.log2(i*h);
}

function fun(h) {
   var sum = 0;
   for (var i = 1 ; i < 5555 ; i++) {
       for (var l = 1 ; l < 5555 ; l++) {
           sum+=(calcc(i,l));
           sum+=(calcc(i,l));
       }
   } 
   return sum;
}

if (cluster.isMaster) {   
  var CPU = 8;    
  var count = 0;

  function messageHandler(msg) {
    console.log(msg);
    count++;
    if (count == CPU)
      console.log('Complete');
  }

  for (let i = 0; i < CPU; i++) {
    var worker = cluster.fork();
    worker.on('message', messageHandler);
  }    
} else if (cluster.isWorker) {
  process.send(fun());
}

Since a cluster is intended for network connections, it can provide some overhead, so low-level implementation with child_process.fork() (which is used by cluster ) could be more efficient.

And there are experimental Node worker threads for Node 10 and higher.

Answer 2

Never used the parallel myself, but based on behaviour and how you use it, I suppose @estus is basically right.

If the function itself is passed -> it does not bring with itself any information about previous surroundings.

You will need to initialize at the function beginning everything you use.

Most direct way is to just put it inside main function:

function fun(h)
{
    const calc = function(){...}
}

The most elegant is to use require inside. So put everything you want to share into another module, export it and just do the

function fun(h)
{
    const tools = require('tools');
    tools.calc(...)
}