Clojure中无限循环中的内存泄漏

Question

I have the following function:

(defn- clock-process [every f]
  (while true
  (f)
  (Thread/sleep every)))

And I'm spinning it off in to its own thread, to avoid blocking the main thread:

(future
  (clock-process 3600000 ;; fire every hour
    #(intensive-fn ...)))

On a Heroku dyno, total memory use is substantially growing at each hour interval:

Where the memory drops off is where the application is restarted.

Can anyone help me understand what's happening?

Answer 1

I don't know enough to help you understand why this is happening, but I can suggest some options for mitigating the problem.

tl;dr: Lowering you heap size is probably the best thing to do.

First, set up the Memory logging agent . This will periodically print some messages to the logs like:

source=web.1 measure.mem.jvm.heap.used=33M measure.mem.jvm.heap.committed=376M measure.mem.jvm.heap.max=376M
source=web.1 measure.mem.jvm.nonheap.used=19M measure.mem.jvm.nonheap.committed=23M measure.mem.jvm.nonheap.max=219M
source=web.1 measure.threads.jvm.total=21 measure.threads.jvm.daemon=11 measure.threads.jvm.nondaemon=1 measure.threads.jvm.internal=9

From this, you'll be able to determine where growth occurs (ie heap or non-heap).

If you find that the growth is all heap, you should try setting your Maximum heap size (ie -Xmx ). lower than the defaults. Something like -Xmx300m should be sufficient, but you can probably go lower. You could also generate some heap dumps, and analyze them with a tool like Eclipse MAT if you want to pinpoint the source.

If you find that growth is in non-heap, then use some of the other steps described in the troubleshooting guide to determine if it's Metaspace or something else. If it is not in Metaspace, then you may have a native memory leak, which can result from leaving file handles or buffers open.

Have you tried to reproduce the problem locally? If you do so, if will be easier to dig a little deeper using tools like VisualVM and jmap .

Finally, you can open a support ticket with Heroku . They can look at things like smaps to help with non-heap/native-memory leaks.