Using rxjs to sort / group on a custom property added to a Firestore collection

Question

I have some comments in a Firestore collection, with this flat structure:

export interface Comment {
  postId: string;
  inReplyTo: string;
  user: string;
  comment: string;
  timestamp: firebase.firestore.FieldValue;
}

Here's some simplified sample data:

[
  {commentId: 'A', inReplyTo: '', timestamp: '2018-03-22'},
  {commentId: 'B', inReplyTo: '', timestamp: '2018-01-15'},
  {commentId: 'C', inReplyTo: 'B', timestamp: '2018-04-14'},
  {commentId: 'D', inReplyTo: 'A', timestamp: '2018-07-08'},
]

The comments are not intended to be deeply / recursively threaded. There are only top-level comments and replies.

I query Firestore for them like this:

this._commentsCollection = this.afs.collection('comments', ref => ref.where('postId', '==', this.postId)
    .orderBy('timestamp'));

Because I need the document ID metadata for the comments, I call SnapshotChanges() and then map() :

this.comments = this._commentsCollection.snapshotChanges().pipe(
    map(actions => {
        return actions.map(a => {
            const data = a.payload.doc.data();
            const id = a.payload.doc.id;

            var topLevelCommentId;
            if ( data.inReplyTo == '' ) {
                topLevelCommentId = id;
            } else {
                topLevelCommentId = data.inReplyTo;
            }

            return { id, topLevelCommentId, ...data };
        });
    })
);

You'll see that I also add a topLevelCommentId property to each comment. My goal is to sort on this property to display the comments in the correct order in my template.

I understand that I shouldn't use pipes with ngFor because of performance concerns.

So, is it possible to instead use RxJS to group and order these the way I want? First, I want to sort the top-level comments by timestamp. Then I want to group them all by topLevelCommentId . Next, I want to sort each group by inReplyTo , so that I end up with an order like this:

[
  {commentId: 'B', inReplyTo: '', timestamp: '2018-01-15'},
  {commentId: 'C', inReplyTo: 'B', timestamp: '2018-04-14'},
  {commentId: 'A', inReplyTo: '', timestamp: '2018-03-22'},
  {commentId: 'D', inReplyTo: 'A', timestamp: '2018-07-08'},
]

In SQL I could say something like ORDER BY timestamp, topLevelCommentId, inReplyTo = '' and it would sort them into the right order for me.

In Javascript it should be pretty straightforward to do as well - if I were dealing with a normal array. Which it sort of is, except it's one that is streamed.

What I don't know how to do, is to hook into the pipe() above to sort the array by topLevelCommentId . Like, where in that chained sequence do I put my code? And, are there any RxJS operators that will make it easier for me?

I've already looked at two related questions, and they do not answer my question. I can't use this one because it'd force me to nest *ngFor async loops in my template, which doesn't work. This other one was more promising, however the given answer did not work for me. I get the error message Property 'groupBy' does not exist on type 'any[]'.

I will be grateful for any suggestions or insights.

Edit: I just realized that simply running them through groupBy would sort them into the right order, assuming that the timestamp order is preserved with that operation. My larger question still stands. Where do I place that code within the pipe() above?

Edit 2: I was able to solve this in a different way. Instead of trying to sort the observable stream, I subscribed to it, converted it to a normal Javascript array, and then used standard Javascript sort methods on it. My template uses ngFor to loop over this.sortedComments now, instead of this.comments. Here's the code. It's probably unnecessarily verbose but I find it easier to follow that way. It also might be more computationally intensive but I am not dealing with a large number of comments so it is sufficient for my purposes.

public sortedComments = [];

// load comments 
this._commentsCollection = this.afs.collection('comments', ref => ref.where('postId', '==', this.postId).orderBy('timestamp'));

// subscribe to the comments observable
this._commentsCollection.snapshotChanges().subscribe(rawData => {

  // convert the array of DocumentChangeAction elements to a normal array
  let normalArray = [];
  let allTopLevelIds = [];
  rawData.forEach( (a) => {

      const data = a.payload.doc.data();
      const id = a.payload.doc.id;

      // Add another property to indicate which top-level thread this
      // comment belongs to
      var topLevelCommentId;
      if ( data.inReplyTo == '' ) {
        topLevelCommentId = id;
      } else {
        topLevelCommentId = data.inReplyTo;
      }

      // Keep a record of all possible top-level IDs for use later
      if ( allTopLevelIds.indexOf(topLevelCommentId) == -1 ) {
        allTopLevelIds.push(topLevelCommentId);
      }

      const thisObj = { id, topLevelCommentId, ...data }

      // push this object onto the normalArray
      normalArray.push(thisObj);
  });

  // Loop over all toplevel Ids
  allTopLevelIds.forEach( (a) => {

    // Loop over all comments and find the actual top-level comment for this thread
    normalArray.forEach( (b) => {

      if ( b.id == a ) {
        this.sortedComments.push(b); // push the first comment in this thread onto the array

        // Loop over all comments again and find all replies to the top-level comment
        normalArray.forEach( (c) => {

          if ( ( c.id != a ) && ( c.inReplyTo == a ) ) {
            this.sortedComments.push(c); // push this reply onto the array
          }

        });

      }
    });

  });

});

Answer 1

Actually, all you need to do is perform a nested sort, and all is good. There is no need to go for groupBy (yeah, the naming slightly different in SQL syntax as compared to JS), because although you may achieve the same result using groupBy , you will need to flattened it back to a your 1 dimensional array.

Simply perform another .map() in your pipe, and this time round use .sort() from Javascript's native API:

map((actions) => {
    return actions.sort((a, b) => {
        //first sort by timestamp
        if (a.timestamp > b.timestamp)
            return -1;
        if (b.timestamp > a.timestamp)
            return 1;

        // if timeStamp is the same, proceed to compare inReplyTo
        if (a.inReplyTo > b.inReplyTo)
            return -1;
        if (b.inReplyTo > a.inReplyTo)
            return 1;

        // if inReplyTo is the same, proceed to compare commentId
        if (a.commentId > b.commentId)
            return -1;
        if (b.commentId > a.commentId)
            return 1;
        return 0;
    })
})

Here is a much, much succinct code which basically does the same as above:

this._commentsCollection.snapshotChanges()
    .pipe(
        map(actions => actions.map(a => ({
                data: a.payload.doc.data(),
                id: a.payload.doc.id,
                topLevelCommentId: a.payload.doc.data().inReplyTo === '' ? a.payload.doc.id : a.payload.doc.data(),
            })
        )),
        map((actions) => actions.sort((a, b) => {                            
                let timestampDiff = a.timestamp > b.timestamp;
                let inReplyToDiff = a.inReplyTo > b.inReplyTo;
                let commentIdDiff = a.commentId > b.commentId;

                return timestampDiff === 0 ? (inReplyToDiff === 0 ? commentIdDiff : inReplyToDiff) : timestampDiff;
            })
        )
    );