How to append a child element to a large XML file using Java?

Question

I am trying to create a XML file, using Java, that is a collection of GPS coordinates (GPX). Every time I receive a coordinate from my android device (approximately 1 every second) I need to append the results to an existing XML file. The output that I am looking for is shown below with the trkpt element as the repeated item. The problem is that I can't just add the new trkpt to the end of the file because it needs to be inside the trkseg parent element.

So far I have tried two different APIs, SIMPLEXML and JDOM. With SIMPLEXML I couldn't figure out how to append a child element to an existing file so I switched to JDOM. JDOM allowed me to append the trkpt element as shown below, but as the file started growing it quickly slowed down the user interface of the program. With JDOM I was using the SAXBuilder to reopen the file and append. I think the issue with this was that it had to reproduce the entire file in memory before it added the new element and rewrote the file. So the larger the file got the more demanding the operation was on the device. I need a solution that doesn't examine/copy the entire file before writing the new data. Is there a more efficient way to accomplish this with Java or an API for Java? Thanks for any help!

<?xml version="1.0" encoding="UTF-8"?>
<gpx xmlns="http://www.topografix.com/GPX/1/1">
        <trk>
            <trkseg>
                <trkpt lon="9.860624216140083" lat="54.9328621088893">
                    <ele>228.0</ele>
                </trkpt>
                <trkpt lon="9.860624216140100" lat="54.9328621088754">
                    <ele>234.0</ele>
                </trkpt>
                <trkpt lon="9.860624216140343" lat="54.9328621088678">
                    <ele>227.0</ele>
                </trkpt>
            </trkseg>
        </trk>
</gpx>

Answer 1

There are always bottlenecks when it comes to I/O, especially when opening/closing/re-opening files in a repetative way.

A DOM handler will create a whole tree structure every time it opens the file, but is very effective when it comes to alter that tree.

So first of all, do you really need to open, alter, save the file on every tick? If not, keep the DOM of the file in memory, and alter through the reference to the XML. Save when the user exits the app or leaves a view.

If you do need to save the file at each tick, you could still keep the DOM in memory and only save it to disk on each tick.

If you need to open/save/re-open the file on each tick, don't use any XML-library - simply use a standard FileWriter or such alike, with manual altering of the contents - but it will still be hard to maintain performance if the file gets really large.

Answer 2

This sounds like the perfect application for SAX (find it in the package org.xml.sax ); it's a streaming API for XML access and manipulation. SAX generates events for every element it encounters, allowing you to copy the file to a new file without having to parse it into a large in-memory tree. When you have reached the end of your input file, just append the new element as appropriate before processig the end tag of <trkseg> .

Of course your approach of rewriting this file every second sounds questionable in and of itself. Can you bundle the information in larger segments? You could dump the information to single files and collect them into a single file at certain intervals (every 10/30/60 seconds).

Answer 3

如果它很简单，则可以使用RandomAccessFile并寻找减去几个字节的文件长度（就在根关闭标记之前），然后开始覆盖。

Answer 4

I suggest to split the xml file into 3 parts.

head.xml

<?xml version="1.0" encoding="UTF-8"?>
<gpx xmlns="http://www.topografix.com/GPX/1/1">
    <trk>
        <trkseg>

body.xml

<trkpt lon="9.860624216140083" lat="54.9328621088893">
    <ele>228.0</ele>
</trkpt>
<trkpt lon="9.860624216140100" lat="54.9328621088754">
    <ele>234.0</ele>
</trkpt>
<trkpt lon="9.860624216140343" lat="54.9328621088678">
    <ele>227.0</ele>
</trkpt>

tail.xml

        </trkseg>
    </trk>
</gpx>

now whenever you get new data, simply append it to body.xml

to read xml file use SequenceInputStream as below:

List<InputStream> list = new ArrayList<InputStream>(3);
list.add(new FileInputStream("head.xml"));
list.add(new FileInputStream("body.xml"));
list.add(new FileInputStream("tail.xml"));
InputStream xmlStream = new SequentialInputStream(Collections.enumeration(list));