Processing huge amount of large jpgs

Question

I am writing a program partly just for fun, partly to help me deal with a heap of digital pictures that I want to divide into categories for printing. The main idea is that it should display the pictures in a single column and have a set of checkboxes, with category names, next to each picture. I check the needed checkboxes, press the "go!" button, and the pictures get copied into subfolders, depending on the selected checkboxes.

Now, everything is almost finished - except for one thing. The pictures in question are large jpgs, each about 7-8MB, and there's approximately 700 of them. If I try to load them all at once, naturally, it takes a huge amount of memory and time to load them all. So, is there a good solution to the problem? My two thoughts were as follows.

1) To load the pics 10 at a time and have next/previous buttons somewhere. I don't like this idea, as it adds unneeded elements. 2) To make the application load new pictures as you scroll to them and unload those you scrolled past. I really like this idea.

Can someone point me in the right direction, as to how I can implement the latter idea? I have only found one relevant link on Google, but I cannot say that it helped me, i got a bit confused by some parts of the code.

Answer 1

If thumbnails are sufficient, this answer includes a simple approach to resampling and cites some trade-offs. If not, this answer outlines a general approach to displaying and caching recent images.

In either case, the default Boolean renderer/editor of JTable is a JCheckBox . CheckOne is an example.

Answer 2

You have to create thumnails for all pictures, you may keep the thumnails in memory. That may need a lot of time.

Then you are either ready with the pronblem. or the thumbnail will not fit all in memory. If that is the case: you load 30-40 of them, and during scrolling you detct the scroll direction, and load the next bunch in a separate thread.

If the loading is slower than the user scrolls, then you dispaly a default place holder image for such a "not yet loaded pic"

Answer 3

You can shrink down all your images to use less RAM. eg this code reduces all images to 200x200. That way you can fit 1000 images in 100MB.

import javax.swing.*;
import java.awt.*;
import java.awt.image.*;
import javax.imageio.ImageIO;
import java.io.File;

public class Scroll extends JPanel {
    public static void main(String[] args) throws Exception {
        JFrame frame = new JFrame();
        JPanel panel = new Scroll();

        panel.setLayout(new BoxLayout(panel, BoxLayout.Y_AXIS));

        for(int i = 0; i < 10; i++) {
            JPanel buttonPanel = new JPanel();
            JRadioButton b1 = new JRadioButton("button 1");
            JRadioButton b2 = new JRadioButton("button 2");
            JRadioButton b3 = new JRadioButton("button 3");
            ButtonGroup group = new ButtonGroup();
            group.add(b1);
            group.add(b2);
            group.add(b3);
            buttonPanel.add(b1);
            buttonPanel.add(b2);
            buttonPanel.add(b3);

            BufferedImage buffer = new BufferedImage(200,200,BufferedImage.TYPE_INT_RGB);
            Graphics2D g = buffer.createGraphics();

            BufferedImage image = ImageIO.read(new File("image.jpg"));
            g.scale(buffer.getWidth()*1.0/image.getWidth(),
                    buffer.getHeight()*1.0/image.getHeight());
            g.drawImage(image, 0, 0, null);
            g.dispose();
            JLabel imageLabel = new JLabel(new ImageIcon(buffer));
            JSplitPane splitPane = new JSplitPane();
            splitPane.add(imageLabel, JSplitPane.LEFT);
            splitPane.add(buttonPanel, JSplitPane.RIGHT);
            panel.add(splitPane);
        }
        JScrollPane spane = new JScrollPane(panel);
        frame.add(spane);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setSize(500,600);
        frame.setVisible(true);
    }
}

If you want to dynamically load images as they become visible you have to use empty JPanels for each image instead of ImageIcons and then you override the paint method of those JPanels. The paint method will only be called if the JPanel is visible. So the simplest solution would be to always load the image from disk in the paint method and then draw it to the screen.

import javax.swing.*;
import java.awt.*;
import java.awt.image.*;
import javax.imageio.ImageIO;
import java.io.File;

public class Scroll extends JPanel {
    public static void main(String[] args) throws Exception {
        JFrame frame = new JFrame();
        JPanel panel = new Scroll();

        panel.setLayout(new BoxLayout(panel, BoxLayout.Y_AXIS));

        for(int i = 0; i < 10; i++) {
            JPanel buttonPanel = new JPanel();
            JRadioButton b1 = new JRadioButton("button 1");
            JRadioButton b2 = new JRadioButton("button 2");
            JRadioButton b3 = new JRadioButton("button 3");
            ButtonGroup group = new ButtonGroup();
            group.add(b1);
            group.add(b2);
            group.add(b3);
            buttonPanel.add(b1);
            buttonPanel.add(b2);
            buttonPanel.add(b3);

            JPanel imagePanel = new JPanel() {
                {
                    BufferedImage image = ImageIO.read(new File("image.jpg"));
                    setPreferredSize(new Dimension(image.getWidth(), image.getHeight()));
                    image.flush();
                }
                @Override
                public void paint(Graphics g) {
                    try {
                        BufferedImage image = ImageIO.read(new File("image.jpg"));
                        g.drawImage(image, 0, 0, null);
                        image.flush();
                    } catch(Exception e) {
                    }
                }
            };

            JSplitPane splitPane = new JSplitPane();
            splitPane.add(imagePanel, JSplitPane.LEFT);
            splitPane.add(buttonPanel, JSplitPane.RIGHT);
            panel.add(splitPane);
        }
        JScrollPane spane = new JScrollPane(panel);
        frame.add(spane);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setSize(500,600);
        frame.setVisible(true);
    }
}

Answer 4

If this is a "use once and throw away" kind-of program I wouldn't bother trying to solve this problem. I would use the command line to generate thumbnails for each of the pictures, and use those generated thumbnails in my program. Keeping 700 thumbnails in memory should be doable.

If that is not an option, I would just start with a JTable . You can create a TableModel without having to load all the images into memory. You just need to know the image count. The JTable will only want to render the images which are visible at that moment. Caveat might be that loading the images at the moment the renderer asks for them might be too late. I can imagine that the JTable will not run smoothly when you have to load images with a size of a few MB's. But this can probably be solved by using a cache which gets populated with a worker thread. So if the renderer request the 5th image, you also load the 5 previous images and the 5 next images into the cache.

Also note that if you are moving the pictures iso copying them, this might affect your TableModel as the number of images in your directory will change. Make sure to consider that !