Best Practice for Callback Parameters in NodeJs

Question

Background

I'm the maintainer of a low level library for fast object traversal in Node.js. The focus of the library is speed and it is heavily optimised. However there is one big slowdown: Callback Parameters

The Problem

Callbacks are provided by the library consumer and can be invoked many, many times per scan. For every invocation all parameters are computed and passed to the callback. In most cases only a fraction of the parameters are actually used by the callback.

The Goal

The goal is to eliminate the unnecessary computation of these parameters.

Solutions Ideas

• Ideally NodeJs would expose the callback parameters as defined by the callback. However obtaining them doesn't seem to be possible without a lot of black magic (string parsing). It would also not solve the situation where parameters are only required conditionally.
• Instead of trying to obtain the parameters from the callback, we could require the callback to expose the required parameters. It sounds very inconvenient and error prone and would also not solve conditionally requires.
• We could introduce a different callback for every parameter combination. This sounds like a bad idea.
• Instead of passing in the parameters directly, we could pass in a function for each parameter that computes and returns the parameter value. Inside the callback the parameter would then be invoked as required. It's ugly but might be the best approach?

Questions

• How do other libraries solve this?
• What are other ways this can be solved?

This is a very fundamental design decision and I'm trying to get this right.

Thank you very much for your time! As always appreciated!

Answer 1

You could pass to the callback an object that has various methods on it that the client using the callback could call to fetch whatever parameters they actually need. That way, you'd have a clean object interface and you'd only compute the necessary information that was actually requested.

This general design pattern is sometimes called "lazy computation" where you only do the computation as required. You can use either accessor functions or getters, depending upon the type of interface you want to expose.

For performance reasons, you can perhaps reuse the same object for each time you call the callback rather than building a new one (depends upon details of your implementation).

Note that you don't even have to put all the information needed for the computation into the object itself as the methods on the object can, in some cases, refer to your own local context and locally scoped variables when doing their computation.

Answer 2

However there is one big slowdown: Callback Parameters

Did you actually benchmark this? I doubt constructing the argument values is that costly. Notice that if this is a really heavily used call, V8 might be able to inline it and then optimise away unused argument values.

Ideally NodeJs would expose the callback parameters as defined by the callback.

Actually, it does . If you do want to rely on this property though, you should properly document that you do, otherwise this magic could lead to obscure bugs.

We could introduce a different callback for every parameter combination. This sounds like a bad idea.

It doesn't seem to be that much of a problem to provide two options, filter(key, value) and filterDetailed(key, value, context) . If the optimisation is really worth it, and as you say this is a low-level library, just go for it.

Instead of passing in the parameters directly, we could pass in a function for each parameter that computes and returns the parameter value. Inside the callback the parameter would then be invoked as required. It's ugly but might be the best approach?

Constructing a closure object to pass instead of a parameter does have some overhead as well, so you will need to benchmark this properly. It might not be worth it.

However, I see that you are actually passing a single context object as the argument on which the computed values are accessed as properties. In that case, you can simply make these properties getters that will compute the value when they are accessed, not when the object is constructed.