Sorting a threaded conversation

Question

Not sure how to frame this one, but here goes.

It's a sorting algorithm question

Tools I have to play with are PostgresSQL and python (2.6) on the server and javascript/jquery at the browser end

I need to move data that describes a threaded conversation from a postgres DB to a web page. The data starts off in chronological order, I want to display in threads

Record numbers are small - should never be more than 100 messages returned

So, as a simple example, imagine the table:

ID     Reply_To   Text
===    ========   =============================
1      0          Hello World
2      0          Make me a sandwich
3      1          Hello back
4      2          No chance
5      1          Who are you?
6      5          Me of course

The end point I want to reach is

• Hello World
• Hello Back
• Who are you?
• Me of course
• Make me a sandwich
• No chance

Or, to put it another way...

ID     Reply_To   Text
===    ========   =============================
1      0          Hello World
3      1          Hello back
5      1          Who are you?
6      5          Me of course
2      0          Make me a sandwich
4      2          No chance

I'm not after a full solution here, all the ajax, json and formatting stuff I'm happy to get on with.

I'm just having issues getting my head around the neatest way to manage the sort.

SQL? Python? Javascript?

I'm currently playing with array sorts in Javascript (for no better reason than the fact that my python skills are exceptionally weak)

EDIT At the moment I'm at something like:

function byThread(a,b) {
  if (a.reply > b.id && a.reply != 0){
    console.log("Compared id=" + a.id + " with id=" + b.id + " and returned -1 ")
    return -1;
  }
  if (a.id > b.reply && b.reply != 0 ){
    console.log("Compared id=" + a.id + " with id=" + b.id + " and returned 1 ")
    return 1;
  }
  console.log("Compared id=" + a.id + " with id=" + b.id + " and returned 0 ")
  return 0;
}
msg.sort(byThread);

And it's frustratingly close

Answer 1

I've tried to do this in pure sql, because I think that's where the logic should belong. What you need to do is find the list of ids from parent to child, and order by that. Luckily, postgres has array types which can be ordered, so we can use a recursive CTE:

with recursive threaded(id, reply_to, message, order_path) as (
    select 
        parent.id, 
        parent.reply_to, 
        parent.message, 
        NULL::int[] || parent.id -- create an 1-element array with the parent id
    from conversation parent
    where parent.reply_to is null
    UNION ALL
    select 
        reply.id, 
        reply.reply_to, 
        reply.message, 
        t.order_path || reply.id -- append the reply id to the current path
    from threaded t
    join conversation reply on t.id = reply.reply_to
    where reply.reply_to is not null
)
select * from threaded order by order_path;

And the results:

1    NULL    "Hello World"          "{1}"
3    1       "Hello Back"           "{1,3}"
5    1       "Who are you?"         "{1,5}"
6    5       "Me of course"         "{1,5,6}"
2    NULL    "Make me a sandwich"   "{2}"
4    2       "No Chance"            "{2,4}"

I'm not sure how this will perform though, so you should definitely test and profile this on your real dataset to make sure it's fine. If it's not, perhaps you could look at restructuring your data, and investigating different ways of storing "tree" data in a database. There is a library for django called django-mptt that can efficiently store and retrieve trees. The concept applies to databases in general, but the algorithms for pulling out trees and making sure they stay intact require changes to your application logic, better handled by a library.

Edit:

I should mention that I was originally using just the top-level id for "order_path" as a single number. This answer led me to using an array of ids to guarantee the order all the way down.

Answer 2

You could try something like this on JS side. Since this have replies inside replies, it'd be easy to build a DOM from the convoSorted

var convo = [{id: 1, replyTo: 0, text: "hello world"}, 
             {id: 2, replyTo: 0, text: "Make me a sandwich"},
             {id: 3, replyTo: 1, text: "hello back"},
             {id: 4, replyTo: 2, text: "no chance"},
             {id: 5, replyTo: 1, text: "who are you?"},
             {id: 6, replyTo: 5, text: "me of course"},
             {id: 7, replyTo: 0, text: "new question"},
             {id: 8, replyTo: 7, text: "new answer"}];

var convoSorted = [];

function addToReplies(obj, rply){ //recursive function to find the q. to reply to.
  if (obj.id == rply.replyTo){
    obj.replies.push({id: rply.id, text: rply.text, replies: []}); //add to the replies array
  }
  else{
      for (k = 0; k < obj.replies.length; k++){
        addToReplies(obj.replies[k], rply);
      }
  }
}

function sortConvo(){

  for (i = 0; i < convo.length; i++){
    if (convo[i].replyTo == 0){ //if it's not a reply, add to sorted array
      convoSorted.push({ id : convo[i].id, text: convo[i].text, replies: [] });
    }
    else{ //it's a reply, find the question it's replying
      for (j = 0; j < convoSorted.length; j++){
          addToReplies(convoSorted[j], convo[i]);
      }
    }
  }
}

sortConvo();
console.log(convoSorted);

Answer 3

I feel an idiot - I've been over-complicating this.

All it needed was a bit of lateral thinking (which this discussion has helped with)

msgs=[
  {id:1,reply:0,msg:'Hello World'},
  {id:2,reply:0,msg:'Make me a sandwich'},
  {id:3,reply:1,msg:'Hello back'},
  {id:4,reply:2,msg:'No chance'},
  {id:5,reply:1,msg:'Who are you?'},
  {id:6,reply:5,msg:'Me of course'}
];
msgs.sort(function(a,b){
  return a.reply - b.reply;
});

var conversation=$("<div id='threaded'>");
var list = $("<ul>")
  .attr("id","thread_" + msgs[0].reply);
var message = $("<li>")
  .text(msgs[0].msg)
  .attr("id","msg_" + msgs[0].id);
$(list).append(message);
$(conversation).append(list);

for (i=1; i<msgs.length; i++){
  var message = $("<li>")
    .text(msgs[i].msg)
    .attr("id","msg_" + msgs[i].id);
  if ($(conversation).find("#msg_" + msgs[i].reply).length == 0){
    $(conversation).find("ul:eq(0)").append(message);
  } else {
    var list = $("<ul>")
      .attr("id","thread_" + msgs[i].id);
    $(list).append(message);
    $(conversation).find("#msg_" + msgs[i].reply).append(list);
  }
}

Sometimes I forget just what a powerful tool the dom is