[英]Using Pipes to read and write binary data in Haskell
I am trying to read and write very many ints in constant memory. 我试图在常量内存中读取和写入很多整数。 I have figured out how to write the ints to memory but have not figured out how to read them back. 我已经想出了如何将内存写入内存,但还没弄清楚如何将它们读回来。
import Control.Lens (zoom)
import System.IO (IOMode(..), withFile)
import Pipes
import qualified Pipes.Prelude as P
import qualified Pipes.ByteString as PB
import qualified Pipes.Parse as P
import qualified Pipes.Binary as P
intStream :: Monad m => Proxy x' x () Int m b
intStream = go (0 :: Int) where
go i = yield i >> go (i + 1)
decoder :: Monad m => Int -> P.Parser P.ByteString m [Int]
decoder n = zoom (P.decoded . P.splitAt n) P.drawAll
main :: IO ()
main = do
withFile "ints" WriteMode $ \h -> do
runEffect $ for intStream P.encode >-> P.take 10000 >-> PB.toHandle h
withFile "ints" ReadMode $ \h -> do
xs <- P.evalStateT (decoder 10000000) (PB.fromHandle h)
print xs
I got the decoder function from the documentation for Pipes.Binary . 我从Pipes.Binary的文档中获得了解码器功能。 However it uses drawAll
which according to the documentation drawAll
is not idiomatic use of Pipes and is provided for testing purposes. 然而,它使用drawAll
,根据文档 drawAll
不是惯用的管道使用,并提供用于测试目的。
My question is how to modify decoder
so that it doesn't use drawAll
and thus does not load all the values of xs
into memory. 我的问题是如何修改decoder
以便它不使用drawAll
,因此不会将xs
所有值加载到内存中。 So instead of printing the list of xs I could P.map print
over a stream of decoded ints
being read from the file. 因此,不是打印P.map print
列表,而是可以通过从文件读取的解码的ints
流P.map print
。
The docs say that decoded
is a lens from a stream of bytes to a stream of decoded values. 文档说decoded
是从字节流到解码值流的镜头。 We can get the latter out of the former using view
from lens
: 我们可以使用lens
view
从后者中取出后者:
decoder :: Monad m => Int -> Producer P.ByteString m a -> Producer Int m ()
decoder n p = void (view P.decoded p) >-> P.take n
main :: IO ()
main = do
withFile "ints" WriteMode $ \h -> do
runEffect $ for intStream P.encode >-> P.take 10000 >-> PB.toHandle h
withFile "ints" ReadMode $ \h -> do
runEffect $ decoder 10000 (PB.fromHandle h) >-> P.print
I don't have much experience with pipes
, I just followed the types here. 我没有太多pipes
经验,我只是按照这里的类型。 The program seems to function as intended though. 该程序似乎按预期运行。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.