How to properly add elements to ConcurrentHashMap in multiple threads using ExecutorService

Question

I'm trying to load images from some folder to ConcurrentHashMap using multiple threads to save time. Unfortunatelly, some threads 'getting stuck' while they are trying load and put image to my map. As a result, when calling shutdown() program goes further even if some threads didnt perform their tasks. When I set the ExecutorService threads pule to 1, everything goes properly but I waste a lot of time waiting to load all images. It seems to me that there is some race problems but as I know ConcurrentHashMap is safe for multithread operations. I'm still a beginner so please let me understand where is the problem and what I'm doing badly. Here is the code:

public abstract class ImageContainer {

private final static Map<String, BufferedImage> imageMap = loadImages();
private static long loadingTime;

public static Map<String, BufferedImage> loadImages() {

    loadingTime = System.currentTimeMillis();
    ConcurrentHashMap<String, BufferedImage> imageMap = new ConcurrentHashMap<>();
    ExecutorService es = Executors.newFixedThreadPool(5);
    File imageDirectory = new File("Images/");
    if (!imageDirectory.isDirectory()) {
        System.out.println("Image directory error");
    }
    File[] files = imageDirectory.listFiles();
    if (files != null) {
        for (File file : files) {
            if (file.isFile()) {
                es.submit(new Runnable(){
                    @Override
                    public void run() {
                        try{
                            if(file.getAbsolutePath().contains(".jpg")) {
                                imageMap.put(file.getName().replace(".jpg",""),ImageIO.read(file));
                            }
                            else if (file.getAbsolutePath().contains(".png")) {
                                imageMap.put(file.getName().replace(".png",""),ImageIO.read(file));
                            }
                        }
                        catch (IOException e)
                        {
                            System.out.println("Cannot load image");
                        }
                    }
                });
            }
        }
    }
    else
    {
        System.out.println("Image folder empty!");
    }

    es.shutdown();
    try {
        if(!es.awaitTermination(5L, TimeUnit.SECONDS)) {
            System.out.println("Images did not load successfully!");
            es.shutdownNow();
        }
        loadingTime = System.currentTimeMillis() - loadingTime;
    }
    catch(InterruptedException e) {
        System.out.println("Loading images interrupted!");
    }
    System.out.println(imageMap.size());
    return imageMap;
}
};

Answer 1

The problem has most likely nothing to do with ConcurrentHashMap . Each time, you put something in the map, no other thread will be able to put concurrently. So maybe, some threads will have to wait until the other has finished with put , but that will not cause any race conditions.

I executed your code on my machine and everything is working. (No error message, prints the number of loaded images). Maybe your computer is not as fast as mine in loading images and therefor, the awaitTermination times out.

As far as I can tell, I don't know if your approach (loading images with multithreading) is such a good idea. Your harddrive (or SSD) will be the bottleneck and your threads will end up waiting for the hard drive (statement ImageIO.read ). Also, spinning up an executor service (resp. starting new threads) is not very cheap, so maybe you're better of without multithreading. Especially because you only need to load the images once (after, they are cached in the map), so the speedup will probably never be significant. I would consider loading the images sequential.

Answer 2

ImageIO is pretty slow and very I/O intensive so adding many threads often won't help on typical PCs. Are you sure that you just don't need to add a large number for awaitTermination timeout?

Another option is to use LinkBlockingQueue with a limited length for the thread pools so that your main application thread slows down when the consumers are slow. That means the time delay 5L seconds at the end is realistic to allow calls in progress to end.

See JDK source for newFixedThreadPool(n), try a qSize = say 2 or 3 x nthreads in the contructor to LinkedBlockingQueue()

public static ExecutorService newFixedThreadPool(int nThreads) {
    return new ThreadPoolExecutor(nThreads, nThreads,
                                  0L, TimeUnit.MILLISECONDS,
                                  new LinkedBlockingQueue<Runnable>());
}