[英]Understanding core.async merge, in Clojure vs ClojureScript
我與實驗core.async
上的Clojure和ClojureScript,試圖了解如何merge
工作。 特別是, merge
是否使輸入通道上的任何值都可立即用於合並通道。
我有以下代碼:
(ns async-merge-example.core
(:require
#?(:clj [clojure.core.async :as async] :cljs [cljs.core.async :as async])
[async-merge-example.exec :as exec]))
(defn async-fn-timeout
[v]
(async/go
(async/<! (async/timeout (rand-int 5000)))
v))
(defn async-fn-exec
[v]
(exec/exec "sh" "-c" (str "sleep " (rand-int 5) "; echo " v ";")))
(defn merge-and-print-results
[seq async-fn]
(let [chans (async/merge (map async-fn seq))]
(async/go
(while (when-let [v (async/<! chans)]
(prn v)
v)))))
當我嘗試使用帶有較大seq
async-fn-timeout
時:
(merge-and-print-results (range 20) async-fn-timeout)
對於Clojure 和 ClojureScript,我都得到了我期望的結果,因為結果會立即開始打印,並帶有預期的延遲。
但是,當我嘗試使用相同seq
async-fn-exec
時:
(merge-and-print-results (range 20) async-fn-exec)
對於ClojureScript,我得到了我期望的結果,因為結果會立即開始打印,並帶有預期的延遲。 但是對於Clojure而言,即使sh
進程是同時執行的(取決於core.async
線程池的大小),結果似乎最初是延遲的,然后幾乎一次全部打印! 我可以通過增加seq的大小來使這種區別更加明顯,例如(range 40)
由於async-fn-timeout
的結果在Clojure和ClojureScript上都是預期的,因此,將矛頭指向exec
的Clojure和ClojureScript實現之間的差異。
但是我不知道為什么這種差異會導致此問題?
筆記:
async-merge-example.exec
的源代碼 exec
,由於Clojure / Java和ClojureScript / NodeJS之間的差異,Clojure和ClojureScript的實現有所不同。 (ns async-merge-example.exec
(:require
#?(:clj [clojure.core.async :as async] :cljs [cljs.core.async :as async])))
; cljs implementation based on https://gist.github.com/frankhenderson/d60471e64faec9e2158c
; clj implementation based on https://stackoverflow.com/questions/45292625/how-to-perform-non-blocking-reading-stdout-from-a-subprocess-in-clojure
#?(:cljs (def spawn (.-spawn (js/require "child_process"))))
#?(:cljs
(defn exec-chan
"spawns a child process for cmd with args. routes stdout, stderr, and
the exit code to a channel. returns the channel immediately."
[cmd args]
(let [c (async/chan), p (spawn cmd (if args (clj->js args) (clj->js [])))]
(.on (.-stdout p) "data" #(async/put! c [:out (str %)]))
(.on (.-stderr p) "data" #(async/put! c [:err (str %)]))
(.on p "close" #(async/put! c [:exit (str %)]))
c)))
#?(:clj
(defn exec-chan
"spawns a child process for cmd with args. routes stdout, stderr, and
the exit code to a channel. returns the channel immediately."
[cmd args]
(let [c (async/chan)]
(async/go
(let [builder (ProcessBuilder. (into-array String (cons cmd (map str args))))
process (.start builder)]
(with-open [reader (clojure.java.io/reader (.getInputStream process))
err-reader (clojure.java.io/reader (.getErrorStream process))]
(loop []
(let [line (.readLine ^java.io.BufferedReader reader)
err (.readLine ^java.io.BufferedReader err-reader)]
(if (or line err)
(do (when line (async/>! c [:out line]))
(when err (async/>! c [:err err]))
(recur))
(do
(.waitFor process)
(async/>! c [:exit (.exitValue process)]))))))))
c)))
(defn exec
"executes cmd with args. returns a channel immediately which
will eventually receive a result map of
{:out [stdout-lines] :err [stderr-lines] :exit [exit-code]}"
[cmd & args]
(let [c (exec-chan cmd args)]
(async/go (loop [output (async/<! c) result {}]
(if (= :exit (first output))
(assoc result :exit (second output))
(recur (async/<! c) (update result (first output) #(conj (or % []) (second output)))))))))
您的Clojure實現在單個線程中使用阻塞IO。 您首先從stdout中讀取,然后在循環中讀取stderr。 兩者都執行阻塞的readLine
因此它們僅在實際完成讀取一行后才返回。 因此,除非您的進程向stdout和stderr創建相同數量的輸出,否則一個流最終將阻塞另一個流。
一旦該過程完成, readLine
將不再阻塞,並且在緩沖區為空時僅返回nil
。 因此,循環僅完成讀取緩沖的輸出,然后最終完成對“所有一次”消息的解釋。
您可能需要啟動第二個線程來處理從stderr讀取的內容。
node
不會阻止IO,因此默認情況下所有操作都是異步的,並且一個流不會阻止另一個。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.