为要插入数据库的数据实现Queue

Question

We are developing a vehicle tracking system in which several GPS devices keep sending their GPS locations to the server using TCP connection. The TCP communicator decodes the GPS location and inserts that data into the database. Right now, one thread of TCP communicator serves one device request. After decoding the GPS data, it creates a connection to the database and inserts the data and releases the connection. As number of devices are increasing, the number of concurrent connections to the database (which is MySQL) are also increasing. Hence, I want to implement a queue where each thread of TCP communicator will push the data to one end and one job will take data from other end and keep it inserting into the database.

Can anybody suggest me the best solution to handle all this? Our application is based on Java and database is MySQL.

Thanks, Saurabh

Answer 1

您可以使用类似ConcurrentLinkedQueue的线程安全队列实现来对数据进行排队

Answer 2

You could just create a simple Thread that handles the database writes. Then have your communicator threads queue the data that needs to be written with it. Something like this:

public class DatabaseQueue extends Thread {
    private LinkedBlockingQueue<Data> queue = new LinkedBlockingQueue<Data>();

    public void queueData(Data data) {
        queue.add(data);
    }

    public void run() {
        while (true) {
            Data data = queue.take();
            // write data to database
        }
    }
}

Answer 3

A plain concurrent queue is best if you are going to batch your data and it support easy batching techniques (and therefor database insert performance)

However a more flexible approach if you want to do other things as well is to use an ExecutorService with a fixed numebr of threads. This way you can add tasks to do anything, to a limited degree of concurrency.

Answer 4

You need to implement a message queue.

Have a look at either RabbitMQ or ActiveMQ .

Answer 5

one job will take data from other end and keep it inserting into the database.

ConcurrentLinkedQueue is just an overkill, it is for more than one thread to access lock free.The poll method take unnecessary sleep intervals,also u need to wait till the queue is filled , again another kind of sleep. If you only have one thread putting stuff into the queue, and another thread taking stuff out of the queue

Queue<GPRSObj> queue = Collections.synchronizedList(new LinkedList<GPRSObj>());

or

LinkedBlockingQueue