[英]IO Operations in Conduit?
This is probably a pretty obscure case, but please bear with me. 这可能是一个非常晦涩的案例,但请耐心等待。
I'm writing a web crawler that needs to download webpages and save them to local disk. 我正在编写一个网络爬虫,需要下载网页并将其保存到本地磁盘。 Meanwhile, being a crawler, it needs to find the hyperlinks on each page and download those linked pages as well.
同时,作为爬虫,它需要在每个页面上找到超链接并下载那些链接的页面。
Right now I'm looking at the first sample code given on this page . 现在,我正在看此页面上给出的第一个示例代码。 I'm trying to write a conduit of this type:
我正在尝试编写这种类型的管道:
processLink :: Conduit S.ByteString IO S.ByteString
And then I will replace this line in the sample code: 然后,我将在示例代码中替换此行:
responseBody res $$+- sinkFile "foo.txt"
with this line: 用这一行:
responseBody res $= processLinks $$+ sinkFile
So what I'm trying to do is to implement a conduit called processLink, which processes a ByteString, extracks links from them, and then pass the original ByteString along to sinkFile so that the page can be saved. 因此,我想做的是实现一个称为processLink的管道,该管道处理ByteString,从中提取链接,然后将原始ByteString传递给sinkFile,以便可以保存页面。 Then I can call functions on those links recursively to download those linked pages.
然后,我可以递归地调用那些链接上的函数以下载那些链接的页面。
However, since processLink is a pure function, it seems like I can't do any IO operations inside processLink, and therefore cannot do the things I need like downloading those linked pages. 但是,由于processLink是一个纯函数,似乎我无法在processLink内执行任何IO操作,因此无法完成我需要的操作,例如下载那些链接的页面。 Or can I?
可以吗 What could I do to get around this problem?
我该怎么办才能解决这个问题?
Alright I figured it out.... I think what I need is just liftIO lol. 好吧,我知道了...。我想我需要的只是liftIO大声笑。 Sorry I'm kinda new to Haskell :P
抱歉,我是Haskell的新手:P
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.