Loading large JSON file in C#

Question

1) I need to download a large JSON file on a Windows Phone application, I would like to process the JSON as it is downloading, because otherwise the app just keeps downloading the JSON and this can take up to 10 seconds or more, is there any way to do this? I have searched for this for 2 days now and nothing seems to be working...

2) Is there any easy way to check for new items in this JSON file? It's hosted online so I would have to check if there are new items (nodes) added to the file.

Answer 1

1. Just ask a server if there is something new, and make him send to you only actual new stuff. It will be much less, so less data to trasmit on wire, and less time to process and visualize (if need) it.

2. Or create JSON request, that gets data in packets, and after every packet arrive, process it. It will take longer to download, as during processing you will block request, but it will give feedback to a user. You can leverage this, by pushing packet in local JS stack and continue request for another packet, while processing the that one already available on top of it. But it becomes fairly complex scenario, imo, for this simple enough (seems to me) stuff.

Answer 2

1. If you want to parse through a JSON object gradually, look into JSON.NET's JsonReader .

Be aware that on Windows Phone, you can't pull from a network stream on the UI thread, but you can if you push your work to a background thread (have a look at BackgroundWorker ).

2. If your web server supports it, you could pass an If-None-Match / If-Modified-Since header and accept the 304 response to indicate that nothing has changed.

However, this only matches entire HTTP requests. If you need finer detail, you could split your requests into:

a) Request a list of IDs and last modified dates

b) Request the details of a list of IDs (determined by the app based on the modified dates)