将来自Web服务的大量数据分块写入文件

Question

I'm trying to solve the general situation of receiving a big chunk of data from web services via:

-(void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data

For small data, I just use:

[webData appendData:data];

and at the end, in:

-(void)connectionDidFinishLoading:(NSURLConnection *)connection

I write it to my file using an output stream.

What about when the data is huge, a couple of MBs of data? In this case, it would be better to dump data in connection:didReceiveData: several times to the same file. It sounds easy, using NSFileHandle , but syncing for read and write while considering async/non-blocking UI is not straightforward (at least for me).

Any idea what would be a good approach for this case? I was thinking to use NSThread , queueing the chunk size each time. Is there any better/easier approach for this?

Answer 1

You know, I actually made a class for this a while back (non-ARC, but it still should work) available here

Example usage:

// in init
RJRStreamWriter myWriter = [[RJRStreamWriter alloc] initWithLocalFile:@"path/to/my/file" andAppend:NO];

-(void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
    static dispatch_queue_t asyncQueue;
    static dispatch_once_t onceToken;

    dispatch_once(&onceToken, ^{
           asyncQueue = dispatch_queue_create("asyncQueue", NULL);
    });

    dispatch_async(asyncQueue, ^{
        // this method is optimized, and aggregates NSOutputStream, NSMutableData, and NSFileHandle
        [myWriter writeData:data];
    });
}

Answer 2

For a start, a couple of MBs is not huge. Doing it the same way for a couple of MBs would be absolutely fine. But if you do want to write to the file as you receive the data then you could use an NSFileHandle . Create one and then write data to it and then call synchronizeFile on it when you want to flush the data to disk. You could call synchronizeFile after say every 10MB have been written or something (and obviously at the end as well).

So eg:

NSFileHandle *fileHandle = [NSFileHandle fileHandleForWritingAtPath:filename];
...
[fileHandle writeData:dataComeInFromURLConnection];
...
[fileHandle synchronizeFile];