iOS Efficiently Parse Large JSONs from Documents Dir

Question

When I load my app I download around 10-15 JSON files and store them into my apps documents directory-- some range from a few KBs to 30MB.

Once that is finished, I need to grab each of them from the documents dir, convert to a NSDictionary, and parse into NSManagedObjects.

But, when I do that with the code below, as it goes though each JSON it seems to keep them in memory, until the app ends up crashing. Instruments shows nothing in the 'Leaks' tool, but my app is keeping a ton in memory.

Heres the code that grabs the JSON files:

UPDATED

- (void)parseDownloadedFiles
{
    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
    NSString *docDir = [paths objectAtIndex:0];
    docDir = [docDir stringByAppendingString:@"/jsons"];
    NSFileManager *fileManager = [NSFileManager defaultManager];
    NSError *error;   
    NSArray *files = [fileManager contentsOfDirectoryAtPath:docDir error:&error];
    if (files == nil) {
        // error...
    }

for (NSString *file in files) 
{

    NSString *fileName = [NSString stringWithFormat:@"%@/jsons/%@",
                          docDir, file];
    NSString *content = [[NSString alloc] initWithContentsOfFile:fileName
                                                    usedEncoding:nil
                                                           error:nil];

    NSDictionary *JSON =
    [NSJSONSerialization JSONObjectWithData: [content dataUsingEncoding:NSUTF8StringEncoding]
                                    options: NSJSONReadingMutableContainers
                                      error: &error];


       ...create my NSManagedObjects & store

       JSON = nil;
    }  
}

--

Heres a look at my allocations:

--

Drilling into that first Malloc 44.79 brings shows me these problem lines:

-- This is within the for loop in the code above

Would that NSLog really cause such an issue?

Answer 1

Yep, your getContentFromFile... method is returning a retained string. And you never release it on the receiving end.

You need to either autorelease the string when you return it or explicitly release if after you've parsed it into JSON.

(I'd think Analyzer would have found this.)

Answer 2

You should put @autoreleasepool {} inside that loop where you read the file. The objects are not being released until the method returns, so the memory will build up inside the loop.

ARC will help you by autoreleasing the objects but you need them to release faster.

https://developer.apple.com/library/mac/#documentation/Cocoa/Conceptual/MemoryMgmt/Articles/mmAutoreleasePools.html