Efficient storage of large amounts of data in iOS

Question

I'm building an application which has a "record" feature which records user interaction over time. As time progresses, I fill an array in memory with "state" objects representing the current state of the user input. A typical recording will result in about 5k of these objects.

I then archive this data using NSKeyedArchiver archiveRootObject: toFile: . This works fine, however the file size is very large (3.5 megs or so). My question is this:

Is there any inherent file-size overhead involved in archiving files? Would I be able to save this data using much less disk space if I were to use SQLite, or even roll my own file format? Or is the only way to reduce the disk size of the data going to be to reduce the bit depth of the numbers I'm storing?

Answer 1

If your concern is performance, Core Data gives you more granularity. You can lazy load and save by parts during app execution vs loading/saving the whole 3.5Mb object graph.

If your concern is file size, this is the binary plist format , and this is the SQLite file format . But more important than the overhead, is how complex is the translation between your object graph and the Core Data model.

You may also be interested in this comparison of speed and performance for several file formats: https://github.com/eishay/jvm-serializers/wiki/ Not sure if everything there has an C, C++ or objective-C implementation.

Answer 2

3.5 MB isn't a very large file. However, if your app has to load or save a 3.5 MB file all the time, then using Core Data is a lot smarter as this allows you to save only the data that has changed and retrieve only the parts that you're interested in -- not the whole thing every time.

Answer 3

If storage is the main concern, there would be little difference b/w sqlite and core data.

I had to store UIViewControllers with state in an app, where I ended up not saving the serialized objects but saving only the most specific properties and creating a class which read that data and re-allocated those objects.

The property map was then stored in a csv [admittedly very difficult to manage, but small like anything] and then compressed.