收集和过滤大量数据

Question

I have little JAVA coding experience (some classes at university) but none in the area of data collecting.

I want to monitor all the messages send on a CAN bus & store these temporarily to filter them afterwords so I can isolate some specific messages.

This means I have to store a maximum of 17543 messages of 1 byte every second. I would like to store - at least - a couple of tens of seconds.

What the best way to do this?

Can I just store all of these in an array? Would it be better to use a database (such as Hibernate)?

I know this is rather open-ended but I would like to get an idea of what's possible and what's the best direction to handle this.

Answer 1

So, you get (at most) 17,543 messages per second, each of which is only 1 byte, and you'll want to store a couple of tens of seconds?

That's not a lot of data, and you can easily store that in an array in memory. Suppose you want to store the messages of 1 minute, that would be 17,543 x 60 seconds = less than 1 MB of data. That's peanuts on modern computers. A database would be overkill.

Answer 2

Hibernate is not a database. It's an ORM.

20 seconds = 20 * 17543 bytes 
           = 350860 bytes
           = 350 KB 
           = nothing. 

Storing everything in memory wouldn't be a problem for such a small amount of data.

Answer 3

I have to store a maximum of 17543 messages of 1 byte every second. I would like to store - at least - a couple of tens of seconds.

It might make sense to store the messages in a circular buffer . This can be easily implemented using an array.

30 seconds worth of data will require ~500KB of storage.