Improving performance of deserializing from large XML strings

Question

i am parsing a big xml file (1 MB) in my WP7 App. The file is part of the project, so its not loaded through the web. Unfortnuately it takes very long, ca. 3 seconds, to get the content i need. I have read, that the problem is the xml serialization, and its better to go for a binary serialization.

But i have my xml file now, is there any possibility to change the format or something of my xml file, so that the parsing will go faster? I have split it in many parts already, but its not significant faster.

Answer 1

1 megabyte isn't particularly big.

A binary format will be more compact and faster, especially if you write your own rather than using the .net serialisation support, which adds a lot of overhead to the data.

If you want to stick with xml, you can usually improve performance significantly by using a brief, compact format:

• use short names for elements and attributes: eg v rather than vertexentry.
• use self-closing elements with data in attributes rather than cdata or child elements to contain single values. This usually works out more compact.
• if you have a list of simple values, consider using a single string value containing a comma-separated list, rather than lots of individual elements/attributes. eg use p="12,22" rather than x="12" y="22". This is less data to read, fewer items to parse, and halves the number of method calls to read values from the xml element/reader.
• only store useful precision. A double converted to a string uses a lot of digits. If you only need 3 decimal places of accuracy, only store 3d.p.

Profile and optimise your loading code - you may find bottlenecks that are nothing to do with xml. You may be able to defer some work, or do some data conversion processing on another thread, but beware of introducing big complexity for small gains.

Finally, try different approaches - XmlDocument rather than XmlReader, or a different library, or pre-loading the data into a MemoryStream. You may find improvements can be made there too.

Or just tell your boss it's because you don't have an eight core xeon with a terabyte of fast ssds... :-)

Answer 2

If you don't need all the data at once, one way to handle it is to asynchronously load chunks of data manually (you might need to parse the data manually) and update the UI in chunks as it loads.

Also, if there is any extra data in the serialization, you could always come up with your own xml schema that is less verbose and only contains the bare information that you need.

Answer 3

You have at least four options:

• Use fastest available XML deserializer. You can find here good comparsion. sharpSerializer seems to fastest.
• You can write your own binary deserializer and serializer.
• You can use SQL CE database engine.
• You can put data on web server and expose web service to query data. But now you have question how to store data on server. Server is usually of course faster than phone but...