Stackoverflow in Common Lisp Mergesort

Question

I've written my first ever Common Lisp function and I'm having trouble tracking down where my error is being produced. How do I trouble shoot the following error:

Error: Stack overflow on value stack. While executing: TRUNCATE

Here is my code:

(defun mergelist (alist low mid high)
    (setq i1 low)
    (setq i2 (+ mid 1))
    (setq i low)
    (setq blist `())
    (loop while (and (<= i1 mid) (<= i2 high)) do
        (if (<= (nth i1 alist) (nth i2 alist))
            (setf (nth (+ i 1) blist) (nth (+ i1 1) alist))
            (setf (nth (+ i 1) blist) (nth (+ i2 1) alist))
        )
    )
    (loop while (<= i1 mid) do
       (setf (nth (+ i 1) blist) (nth (+ i1 1) alist)) 
    )
    (loop while (<= i2 high) do
        (setf (nth (+ i 1) blist) (nth (+ i2 1) alist))  
    )
    (setq j low)
    (loop for j from j to high do
        (setf (nth i alist) (nth i blist))
    )
)

(defun mergesort (alist low high)
    (when (< low high)
        (mergesort alist low (/ (+ low high) 2))
        (mergesort alist (/ (+ low high) (+ 2 1)) high)
        (mergelist alist low (/ (+ low high) 2) high)
    )
)

The following is how I'm testing the function:

(setq dlist `(5 1 4 2 3))
(mergesort dlist 0 4)

My expected return is:

(1 2 3 4 5)

Answer 1

There are quite a few things we can do to improve this code.

1. Indenting

Lisp has relatively little syntax, but we use indenting to help highlight the structure of the code. Most Lisp aware editors help manage that. The most obvious departure from the conventional indenting approach is closing parentheses on following lines. I've Indented the mergelist function to show a more readable function body - well, at least to me.

(defun mergelist (alist low mid high) 
  (setq i1 low)
  (setq i2 (+ mid 1))
  (setq i low)
  (setq blist `())
  (loop while (and (<= i1 mid) (<= i2 high)) do
       (if (<= (nth i1 alist) (nth i2 alist))
       (setf (nth (+ i 1) blist) (nth (+ i1 1) alist))
       (setf (nth (+ i 1) blist) (nth (+ i2 1) alist))))
  (loop while (<= i1 mid) do
       (setf (nth (+ i 1) blist) (nth (+ i1 1) alist)))
  (loop while (<= i2 high) do
       (setf (nth (+ i 1) blist) (nth (+ i2 1) alist)))
  (setq j low)
  (loop for j from j to high do
       (setf (nth i alist) (nth i blist))))

2. Setq's, setf's vs Let.

The code above creates variables in the top level environment by setq'ing them ( unless of course, you have defparametered them elsewhere). This can have some undesirable side effects as programs get larger ( what if two functions use "i" at the same time?) . Its better to use LET to create local lexical variables so something like

(defun mergelist-2 (alist low mid high)
  (let ((i1 low)
    (i2 (+ mid 1)
      i (low)
      blist '()))

   (loop while (and (<= i1 mid) (<= i2 high)) do
       (if (<= (nth i1 alist) (nth i2 alist))
       (setf (nth (+ i 1) blist) (nth (+ i1 1) alist))
       (setf (nth (+ i 1) blist) (nth (+ i2 1) alist))))
  (loop while (<= i1 mid) do
       (setf (nth (+ i 1) blist) (nth (+ i1 1) alist)))
  (loop while (<= i2 high) do
       (setf (nth (+ i 1) blist) (nth (+ i2 1) alist)))
  (setq j low)
  (loop for j from j to high do
       (setf (nth i alist) (nth i blist))) ))

3. Forms can return values

A lisp form often returns a value. If we type in (+ 1 2) at the repl we will see 3. A defun will normally return a value usually as the last form in its body.

If we look at mergelist we see that it is not explicility returning any value but is instead trying to use the variable alist to communicate the return values. This does not work!

Lisp provides the trace facility which lets us understand what is going on inside

This is the first 16 lines of trace. My system craps out at line 600

0: (MERGESORT (5 1 4 2 3) 0 4) 1: (MERGESORT (5 1 4 2 3) 0 2) 2: (MERGESORT (5 1 4 2 3) 0 1) 3: (MERGESORT (5 1 4 2 3) 0 1/2) 4: (MERGESORT (5 1 4 2 3) 0 1/4) 5: (MERGESORT (5 1 4 2 3) 0 1/8) 6: (MERGESORT (5 1 4 2 3) 0 1/16) 7: (MERGESORT (5 1 4 2 3) 0 1/32) 8: (MERGESORT (5 1 4 2 3) 0 1/64) 9: (MERGESORT (5 1 4 2 3) 0 1/128) 10: (MERGESORT (5 1 4 2 3) 0 1/256) 11: (MERGESORT (5 1 4 2 3) 0 1/512) 12: (MERGESORT (5 1 4 2 3) 0 1/1024) 13: (MERGESORT (5 1 4 2 3) 0 1/2048) 14: (MERGESORT (5 1 4 2 3) 0 1/4096) 15: (MERGESORT (5 1 4 2 3) 0 1/8192) 16: (MERGESORT (5 1 4 2 3) 0 1/16384)

at line 600 you see

600: (MERGESORT (5 1 4 2 3) 0 1/1037378892220248239628101965922790287753111558060609224998914332422663202853227036599926762236775948572049471652825197295598787768852943826971718708528490921765295450850377380921344)

This is a very small number and explains the error message about truncate.

You can see that the alist array is not changing as we proceed down the call stack. Thats because the alist function parameter is local to each invocation of mergelist.

What we need to do is have mergelist return a value explicility each time its called.

(defun mergelist-3 (alist low mid high)
  (let ((i1 low)
    (i2 (+ mid 1)
      i (low)
      j
      blist '()))

   (loop while (and (<= i1 mid) (<= i2 high)) do
       (if (<= (nth i1 alist) (nth i2 alist))
       (setf (nth (+ i 1) blist) (nth (+ i1 1) alist))
       (setf (nth (+ i 1) blist) (nth (+ i2 1) alist))))
  (loop while (<= i1 mid) do
       (setf (nth (+ i 1) blist) (nth (+ i1 1) alist)))
  (loop while (<= i2 high) do
       (setf (nth (+ i 1) blist) (nth (+ i2 1) alist)))
  (setq j low)
  (loop for j from j to high do
       (setf (nth i alist) (nth i blist)))
  *;return value here*
  ))

As a further hint , the last loop in the function is not needed.

Additionally, you will have to capture that return value in mergesort and have mergesort return an explicit value as well.

I would also recommend that you read a bit about the loop macro - google for "Practical Common Lisp Loop for Black belts" which will help you grasp the syntax and the sort of things you can do with loop.

Now, there are still a few things to fix in the code but I hope I've given you enough to get through this iteration

Answer 2

There is too much wrong with the code:

• treats lists as vectors: if you want a Lisp data structure with random access, then use a vector. Not a list.
• doesn't declare variables
• while loops don't work at all
• imperative, instead of functional
• wrong code layout

Best to start from scratch and to avoid above mistakes. If you want Lisp lists, then use a different way to write a mergesort. Hint: lists are not like vectors. They might be used like that, but usually it's a mistake to do so.