Which data structure is sensible for time series data in Java?

Question

I'm new to Java, so I'm not sure which choice of data structure would be good here. I'll store accelerometer, gyroscope and magnetometer data (9 values) in a list, to be used later for smoothing, display and some signal processing.

My idea is to create an object MyObject that has ten members: time stamp and the nine orientation/motion values, all of them floats. I'll then store the data in an ArrayList<MyObject> . Is it a good idea or have I overlooked something?

The list will hold at most 100k values.

Answer 1

Use TreeMap for a start to improve lookup performance.

TreeMap

Note (in docs):

This implementation provides guaranteed log(n) time cost for the containsKey, get, put and remove operations.

Answer 2

Creating your own class is the right way to go. Now for storing the objects of that class...

If you need direct lookup (eg, "give me the 5th object") on the objects once you have all of them, use an ArrayList . But if you will just be looping over all of the objects in order, you should consider using a linked list .

The ArrayList class uses a primitive array that it will grow as needed to accommodate the elements you are adding to it. When it grows its internal structure, it needs to allocate a new array and copy over all of the original values. This can be costly (especially with 100K elements!). You can give it an initial size to give it more time before it needs to grow; but if you end up not needing so much space, the ArrayList's internal array can waste a large chunk of memory.

Adding elements to a linked list costs virtually nothing because "growing" is not needed; it just adds another node to the list. But you can't look up items by their index. You would have to start at the first element and loop through the list to the element you want.

Answer 3

You might want to consider using a file output stream to output your data directly, rather than storing it in some data structure:

output = new BufferedWriter(new FileWriter("output.csv"));

while(dataSource.stillHasData())
    output.println(dataSource.getData().toString());

output.close();

Using a BufferedWriter ensures that the program doesn't have to wait for disk writes to happen before it can take the next data, so this would be acceptable for collecting live data (usually).

You should then do something like this in your data class:

public String toString(){
    StringBuilder buf = new StringBuilder();

    buf.append(timeStamp); str.append(',');
    // ...
    // append all the other data

    return buf.toString();
}

This way of doing it has advantage that you can then import it into programs like excel or really just about any program that you would use to process the data.