Java create unique thread per database row

Question

I have the requirement to poll the database every x seconds and pull out records that need to be processed. I will start a thread for each of these rows. The next time I poll the threads may not have completed yet. I don't want to start another thread for the same ID as it is still processing.

A HashMap<String, Callable> seems appropriate for storing the threads and ensuring that only one thread per ID will exist . What I can't figure out is how do I remove the thread after the thread is done?

My understanding is that if I wait on the thread then it will be blocking the main thread and the polling will not continue.

I have tried the following code and it works, but the HashMap will continue to grow as more tasks come in. There is a possibility that the row could be reprocessed later in the day.

HashMap<String, Callable<String>> callables = new HashMap<String, Callable<String>>();
for(int i =0; i < 10; i++)
{
    for(int j =0; j < 10; j++)
    {
        String key = Integer.toString(j);
        if(!callables.containsKey(key))
        {
            callables.put(key, new Callable<String>() {
                public String call() throws Exception {
                    try {
                        Thread.sleep(500);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    System.out.println("Inside task");
                    return "Task Completed";
                }
            });
            try
            {
                callables.get(key).call();
            }
            catch(Exception ex){
                ex.printStackTrace();
            }
        }
        else
            System.out.println("Task skipped: " + j);
    }
}

Answer 1

If your problem is to remove the couple (key, Callable) from your Map when the thread has finished its job. Call remove() at the end of its call() function.

HashMap<String, Callable<String>> callables = new HashMap<String, 
Callable<String>>();
for(int i =0; i < 10; i++) {
    for(int j =0; j < 10; j++) {
        String key = Integer.toString(j);
        if(!callables.containsKey(key)) {
            callables.put(key, new Callable<String>() {
                final String mKey = key; //store the current key
                public String call() throws Exception {
                    try {
                        Thread.sleep(500);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    System.out.println("Inside task");
                    callables.remove(mKey); //Remove the thread from the Map
                    return "Task Completed";
                }
            });
            try {
                callables.get(key).call();
            }
            catch(Exception ex) {
                ex.printStackTrace();
            }
        }
        else
            System.out.println("Task skipped: " + j);
    }
}

Answer 2

You can wrap Callable s inside your own Callable .

HashMap<String, Callable<String>> callables = new HashMap<>();

public void oneOnly(String id, Callable<String> callable) {
    if (!callables.containsKey(id)) {
        // Wrap the callable in my own.
        callables.put(id, new Callable<String>() {
            public String call() throws Exception {
                String result = callable.call();
                // Remove when finished.
                callables.remove(id);
                return result;
            }
        });

    }
}

public void test(String[] args) throws Exception {
    for (int i = 0; i < 10; i++) {
        for (int j = 0; j < 10; j++) {
            String key = Integer.toString(j);
            oneOnly(key, new Callable<String>() {
                public String call() throws Exception {
                    System.out.println("Inside task");
                    try {
                        Thread.sleep(500);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    return "Task Completed";
                }
            });
            try {
                callables.get(key).call();
            } catch (Exception ex) {
                ex.printStackTrace();
            }
        }
    }
}