[英]Handling sum encoding in Streaming libraries
The motivation behind this question is this scenario - we have a stream of values which are represented by a Sum
encoding. 这个问题背后的动机是这种情况 - 我们有一个由
Sum
编码表示的值流。 Let us assume Either ByteString ByteString
where we represent streams of bytes in error and good states respectively. 让我们假设
Either ByteString ByteString
,其中我们分别表示错误和良好状态的字节流。 Now, we have another function which can compress the ByteString
stream. 现在,我们有另一个可以压缩
ByteString
流的函数。 Is it possible to run this function on Either ByteString ByteString
input stream, and compress either one (not just Right
but also Left
in case Left
is yielded instead of Right
). 是否可以在
Either ByteString ByteString
输入流上运行此函数,并压缩任何一个(不仅仅是Right
而且还有Left
,以防Left
产生而不是Right
)。 A compress
function type signature is below (I am using Streaming library): compress
函数类型签名如下(我正在使用Streaming库):
compress :: MonadIO m
=> Int
-- ^ Compression level.
-> Stream (Of ByteString) m r
-> Stream (Of ByteString) m r
Our input stream is of type Stream (Of (Either ByteString ByteString)) mr
. 我们的输入流是
Stream (Of (Either ByteString ByteString)) mr
。 So, is there some kind of transformer function that can run compress
on input stream, and output a stream of type Stream (Of (Either ByteString ByteString)) mr
where both are compressed. 那么,是否存在某种变换器函数可以在输入流上运行
compress
,并输出Stream (Of (Either ByteString ByteString)) mr
类型的Stream (Of (Either ByteString ByteString)) mr
,其中两者都被压缩。
It seems to me that I should write a custom compress
instead, let us say eitherCompress
as follows: 在我看来,我应该编写一个自定义
compress
,让我们说eitherCompress
如下:
eitherCompress :: MonadIO m
=> Int
-- ^ Compression level.
-> Stream (Of (Either ByteString ByteString)) m r
-> Stream (Of (Either ByteString ByteString)) m r
Is that correct? 那是对的吗? If that is the case, what is a good way to write
eitherCompress
using the function below from zstd
library: 如果是这种情况,使用
zstd
库中的以下函数编写eitherCompress
的好方法是什么:
compress :: Int
-- ^ Compression level. Must be >= 1 and <= maxCLevel.
-> IO Result
I have written stream
producers using yield
, but I have implemented them for simple cases where the input is just a source, not a stream. 我已经使用
yield
编写了stream
生成器,但是我已经针对输入只是源而不是流的简单情况实现了它们。 Will very much appreciate help with this problem. 非常感谢这个问题的帮助。
A common trick to solve these cases is to put each branch of the sum in different monadic layers (so there will be two streaming layers) manipulate each layer separately, and then either consume them separately or re-join them in a single layer. 解决这些情况的一个常见技巧是将和的每个分支放在不同的monadic层(因此将有两个流层)分别操作每个层,然后单独使用它们或在单个层中重新连接它们。
First, two auxiliary functions that use maps
to convert to and from the Sum
composition of functors: 首先,两个辅助函数使用
maps
来转换为仿函数的Sum
组合:
toSum :: Monad m
=> Stream (Of (Either ByteString ByteString)) m r
-> Stream (Sum (Of ByteString) (Of ByteString)) m r
toSum = maps $ \(eitherBytes :> x) ->
case eitherBytes of
Left bytes -> InL (bytes :> x)
Right bytes -> InR (bytes :> x)
fromSum :: Monad m
=> Stream (Sum (Of ByteString) (Of ByteString)) m r
-> Stream (Of (Either ByteString ByteString)) m r
fromSum = maps $ \eitherBytes ->
case eitherBytes of
InL (bytes :> x) -> Left bytes :> x
InR (bytes :> x) -> Right bytes :> x
We do this to be able to use the separate
and unseparate
functions. 我们这样做是为了能够使用
separate
和unseparate
的功能。
The actual compression function would be: 实际的压缩功能是:
eitherCompress :: MonadIO m
=> Int
-> Stream (Of (Either ByteString ByteString)) m r
-> Stream (Of (Either ByteString ByteString)) m r
eitherCompress level =
fromSum . unseparate . hoist (compress level) . compress level . separate . toSum
hoist
is used to work over the monadic layer below the topmost one. hoist
用于在最顶层下方的一元层上工作。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.