How can I make this clojure code run faster?

Question

I have a version implemented in Lisp(SBCL) which runs under 0.001 seconds with 12 samples. However this version(in clojure) takes more than 1.1 secs. What should I do for making this code run as fast as original Lisp version?

To make it sure, my numbers are not including times for starting repl and others. And is from time function of both sbcl and clojure.(Yes, my laptop is rather old atom based one)

And this application is/will be used in the repl, not as compiled in single app, so running thousand times before benchmarking seems not meaningful.

Oh, the fbars are like this: [[10.0 10.5 9.8 10.1] [10.1 10.8 10.1 10.7] ... ], which is Open-High-Low-Close price bars for the stocks.

(defn- build-new-smpl [fmx fmn h l c o]
  (let [fmax (max fmx h)
        fmin (min fmn l)
        fc (/ (+ c fmax fmin) 3.0)
        fcd (Math/round (* (- fc o) 1000))
        frd (Math/round (* (- (* 2.0 c) fmax fmin) 1000))]
    (if (and (> fcd 0) (> frd 0))
      [1 fmax fmin]
      (if (and (< fcd 0) (< frd 0))
        [-1 fmax fmin]
        [0 fmax fmin]))))

(defn binary-smpls-using [fbars]
  (let [fopen (first (first fbars))]
    (loop [fbars fbars, smpls [], fmax fopen, fmin fopen]
      (if (> (count fbars) 0)
        (let [bar (first fbars)
              [_ h l c _] bar
              [nsmpl fmx fmn] (build-new-smpl fmax fmin h l c fopen)]
          (recur (rest fbars) (conj smpls nsmpl) fmx fmn))
        smpls))))

================================================

Thank you. I managed to make the differences for 1000 iteration as 0.5 secs (1.3 secs on SBCL and 1.8 on Clojure). Major factor is I should have created fbars as not lazy but as concrete(?) vector or array, and this solves my problem.

Answer 1

You need to use a proper benchmarking library; the standard Clojure solution is Hugo Duncan's Criterium .

The reason is that code on the JVM starts running in interpreted mode and then eventually gets compiled by the JIT compiler; it's the steady-state behaviour after JIT compilation that you want to benchmark and not behaviour during the profiling stage. This is, however, quite tricky, since the JIT compiler optimizes no-ops away where it can see them, so you need to make sure your code causes side effects that won't be optimized away, but then you still need to run it in a loop to obtain meaningful results etc. -- quick and dirty solutions just don't cut it. (See the Elliptic Group, Inc. Java benchmarking article , also linked to by Criterium's README, for an extended discussion of the issues involved.)

Cycling the two samples you listed in a vector of length 1000 results in a timing of ~327 µs in a Criterium benchmark on my machine:

(require '[criterium.core :as c])

(def v (vec (take 1000 (cycle [[10.0 10.5 9.8 10.1] [10.1 10.8 10.1 10.7]]))))

(c/bench (binary-smpls-using v))
WARNING: Final GC required 4.480116525558204 % of runtime
Evaluation count : 184320 in 60 samples of 3072 calls.
             Execution time mean : 327.171892 µs
    Execution time std-deviation : 3.129050 µs
   Execution time lower quantile : 322.731261 µs ( 2.5%)
   Execution time upper quantile : 333.117724 µs (97.5%)
                   Overhead used : 1.900032 ns

Found 1 outliers in 60 samples (1.6667 %)
    low-severe   1 (1.6667 %)
 Variance from outliers : 1.6389 % Variance is slightly inflated by outliers

A really good benchmark would actually involve an interesting dataset (all different samples, preferably coming from the real world).

Answer 2

When I run this with 1000 samples I get an answer in 46 ms, though here are some common tips on making clojure code faster:

• turn on reflection warnings:

  (set! warn-on-reflection true)

• add type hints until the reflection warnings go away, which isn't a problem here

• make it a lazy sequence so you don't have to construct huge sequences in ram, which results in lots of GC overhead. (in some cases this adds overhead though I think it's a decent idea here)

• in cases like this see if incanter can do the job for you (though that may be cheating)

• time the creation of the result excluding the time it takes the repl to print it.