Java FileWriting and Raspberry Pi

Question

I have been making a Java program for a Raspberry PI and I have reached a problem.

The program reads data from a device and saves it on an XML file using DOM. My problem is that as we add information to the XML it becomes incrementally slower to write more information. We're talking about as soon as it has something as insignificant as 10 entries it becomes a huge bottleneck so much so that it eventually takes longer to write the XML than the frequency to read the data off the device.

I'm thinking the problem is that it rewrites the whole file with every new writing it needs to do. So I read the whole XML, add the entry I want and write it all back. I need a way to just add the new entry on the file without it reading all the data.

I looked for a XML API that allowed this but can't find it.

So my questions are:

• I was told DOM is very slow so is this a problem because I'm using DOM? Or is this just a result of slow reading rates from the Raspberry PI.

• Can you recomend a very light weight XML API that allows just adding and reading specific sections of the XML file? Is that what is meant when it's said that Stax stream XML Files?

• I was rewriting everything to json (because I was told it's much faster) but while doing so I'm wondering if my problem won't persist because I will inevitably read the whole file and write the whole file again.

• What is the most lightweight way to append data to a text file? The data will always be added at the end of the file so i'm wondering if just using plain old text files wouldn't be the best solution?

Answer 1

If I understand correctly, you're trying to update an XML file every time some event happens. That is, when an event happens, you:

• Read the XML file into a DOM.
• Modify some element of the DOM.
• Write the XML file to file.

This isn't playing to XML's strengths, and you would be better off using a file format that lends itself to random access writes. Look into JDBM2, or even a lightweight SQL database like H2.

If you really must have an XML file, you could consider reading it in once at the start of your program, keeps the DOM in memory, writing to file (if changed) on a schedule, in a separate thread to the thread updating it.

If you take this approach, use synchronized methods to ensure that the DOM isn't modified during a file write.

Answer 2

I can attest to the speed of JSon with the raspberry pi. Even reading in the entire file and writing out in JSon will be much faster than DOM.

My application reads in >500 JSon strings, and writes about the same amount back in less than a second. If you do not need to update data, then simply opening a file for writing with the tags "ab" will allow you to append to the file, not overwriting the exiting data.

Answer 3

事实证明，对于我正在做的事情而言，XML编写实际上足够快，并且正在减慢整个系统的速度并造成瓶颈，这是我正在写数据库，但是当没有互联网时，连接超时约为2分钟，因此线程将开始堆积，等待超时结束，这样它们便可以将数据写入XML文件。