走树结构时如何避免`shareReplay`

Question

I am trying to rewrite a package manager using RxJS (the PM is pnpm and the PR is here ).

During the rewrite, I used lots of .shareReplay(Infinity) , which I've been told is bad (I am a beginner in reactive programming)

Could someone suggest an alternative how to rewrite code like the following, w/o using .shareReplay(Infinity) :

'use strict'
const Rx = require('@reactivex/rxjs')

const nodes = [
  {id: 'a', children$: Rx.Observable.empty()},
  {id: 'b', children$: Rx.Observable.empty()},
  {id: 'c', children$: Rx.Observable.from(['a', 'b', 'd'])},
  {id: 'd', children$: Rx.Observable.empty()},
  {id: 'e', children$: Rx.Observable.empty()},
]

// I want this stream to be executed once, that is why the .shareReplay
const node$ = Rx.Observable.from(nodes).shareReplay(Infinity)

const children$ = node$.mergeMap(node => node.children$.mergeMap(childId => node$.single(node => node.id === childId)))

children$.subscribe(v => console.log(v))

Answer 1

The groupBy operator should work here. Looking at the PR this might be a gross over-simplification, but here goes:

 
 
 
  
  'use strict' const Rx = require('@reactivex/rxjs') const nodes = [ {id: 'a', children$: Rx.Observable.empty()}, {id: 'b', children$: Rx.Observable.empty()}, {id: 'c', children$: Rx.Observable.from(['a', 'b', 'd'])}, {id: 'd', children$: Rx.Observable.empty()}, {id: 'e', children$: Rx.Observable.empty()}, ] Rx.Observable.from(nodes) // Group each of the nodes by its id .groupBy(node => node.id) // Flatten out each of the children by only forwarding children with the same id .flatMap(group$ => group$.single(childId => group$.key === childId)) .subscribe(v => console.log(v));
 
  

Edit: More difficult than I thought

Ok, so on my second read through I see that this requires more work than I initially thought so it can't be simplified so easy. Basically, you are going to have to choose between memory complexity and time complexity here since there isn't a magic bullet.

From an optimization standpoint, if the initial source is just an Array then you can remove the shareReplay and it will work in the exact same way, because when subscribing to and ArrayObservable the only overhead is going to be iterating through the Array, there isn't really any extra cost to re-running the source.

Basically for this I think you can think of two dimensions, the number of nodes m and the average number of children n . In the speed optimized version you will end up having to run through the m twice and you will need to iterate through "n" nodes. Since you have m*n children the worst case would be that all of them are unique. Meaning you need to do (m + m*n) operations which simplifies to O(m*n) if I am not mistaken. The draw back of this approach is that you need to have both a Map of (nodeId -> Node) and an Map for removing duplicate dependencies.

'use strict'
const Rx = require('@reactivex/rxjs')

const nodes = [
  {id: 'a', children$: Rx.Observable.empty()},
  {id: 'b', children$: Rx.Observable.empty()},
  {id: 'c', children$: Rx.Observable.from(['a', 'b', 'd'])},
  {id: 'd', children$: Rx.Observable.empty()},
  {id: 'e', children$: Rx.Observable.empty()},
]

const node$ = Rx.Observable.from(nodes);

// Convert Nodes into a Map for faster lookup later
// Note this will increase your memory pressure.
const nodeMap$ = node$
  .reduce((map, node) => {
    map[node.id] = node;
    return map;
  });


node$
  // Flatten the children
  .flatMap(node => node.children$)
  // Emit only distinct children (you can remove this to relieve memory pressure
  // But you will still need to perform de-duping at some point.
  .distinct()
  // For each child find the associated node
  .withLatestFrom(nodeMap$, (childId, nodeMap) => nodeMap[childId])
  // Remove empty nodes, this could also be a throw if that is an error
  .filter(node => !!node)
  .subscribe(v => console.log(v));

The alternative is to use an approach similar to yours which focuses on memory pressure reduction at the cost of performance. Note like I said you can basically just remove shareReplay if your source is an Array because all it is doing when it re-evaluates is re-iterate the array. This removes the overhead of the additional Map. Though I think you still would need distinct to remove duplicates. The worst-case runtime complexity of this would be O(m^2*n) or simply O(m^2) if n is small, since you will need to iterate through all children and for each child you will also need to iterate through m again to find the matching node.

const node$ = Rx.Observable.from(nodes);
node$
  // Flatten the children
  .flatMap(node => node.children$)
  // You may still need a distinct to do the de-duping
  .flatMap(childId => node$.single(n => n.id === childId)));

I would say the first option is preferable in almost all cases, but I leave that to you to determine for your use case. It may be that you establish some heuristic that chooses one algorithm over the other in certain circumstances.

Sidenote: Sorry it wasn't as easy, but love pnpm so keep up the good work!