Java渲染网络摄像头的图像占用太多CPU

Question

I'm writing an application which connect and stream webcam video. To do that, I use Sarxos webcam library (link here ) to get default webcam, then use WebcamPanel to draw image. The problem arose when I delivered the app to my customer, they tested it on an old machine and complained the app took too much CPU.

I never noticed that before, and when I tested it again, to my surprise, the app took around 33% CPU, which is too much for a simple app which only connect webcam and draw image with 30 FPS. Here is my programming environment: Windows 7 64bit, CoreI5-4460 CPU (3.2-3.4Ghz), Zotac Geforce GTX 650 Ti, Java 7u45.

I have tested to point out which part took the most CPU, and it is the rendering. If I fetch webcam images only but not draw them, the CPU takes 6-7%, but when I render them, the CPU jumps to 30-33%. I took a look in to WebcamPanel class to see maybe something is wrong with them, but so far I found nothing. The draw method is as below:

    @Override
    public void paintImage(WebcamPanel owner, BufferedImage image, Graphics2D g2) {

        assert owner != null;
        assert image != null;
        assert g2 != null;

        int pw = getWidth();
        int ph = getHeight();
        int iw = image.getWidth();
        int ih = image.getHeight();

        Object antialiasing = g2.getRenderingHint(KEY_ANTIALIASING);
        Object rendering = g2.getRenderingHint(KEY_RENDERING);

        g2.setRenderingHint(KEY_ANTIALIASING, VALUE_ANTIALIAS_OFF);
        g2.setRenderingHint(KEY_RENDERING, VALUE_RENDER_SPEED);
        g2.setBackground(Color.BLACK);
        g2.setColor(Color.BLACK);
        g2.fillRect(0, 0, pw, ph);

        // resized image position and size
        int x = 0;
        int y = 0;
        int w = 0;
        int h = 0;

        switch (drawMode) {
            case NONE:
                w = image.getWidth();
                h = image.getHeight();
                break;
            case FILL:
                w = pw;
                h = ph;
                break;
            case FIT:
                double s = Math.max((double) iw / pw, (double) ih / ph);
                double niw = iw / s;
                double nih = ih / s;
                double dx = (pw - niw) / 2;
                double dy = (ph - nih) / 2;
                w = (int) niw;
                h = (int) nih;
                x = (int) dx;
                y = (int) dy;
                break;
        }

        if (resizedImage != null) {
            resizedImage.flush();
        }

        if (w == image.getWidth() && h == image.getHeight() && !mirrored) {
            resizedImage = image;
        } else {

            GraphicsEnvironment genv = GraphicsEnvironment.getLocalGraphicsEnvironment();
            GraphicsConfiguration gc = genv.getDefaultScreenDevice().getDefaultConfiguration();

            Graphics2D gr = null;
            try {

                resizedImage = gc.createCompatibleImage(pw, ph);
                gr = resizedImage.createGraphics();
                gr.setComposite(AlphaComposite.Src);

                for (Map.Entry<RenderingHints.Key, Object> hint : imageRenderingHints.entrySet()) {
                    gr.setRenderingHint(hint.getKey(), hint.getValue());
                }

                gr.setBackground(Color.BLACK);
                gr.setColor(Color.BLACK);
                gr.fillRect(0, 0, pw, ph);

                int sx1, sx2, sy1, sy2; // source rectangle coordinates
                int dx1, dx2, dy1, dy2; // destination rectangle coordinates

                dx1 = x;
                dy1 = y;
                dx2 = x + w;
                dy2 = y + h;

                if (mirrored) {
                    sx1 = iw;
                    sy1 = 0;
                    sx2 = 0;
                    sy2 = ih;
                } else {
                    sx1 = 0;
                    sy1 = 0;
                    sx2 = iw;
                    sy2 = ih;
                }

                gr.drawImage(image, dx1, dy1, dx2, dy2, sx1, sy1, sx2, sy2, null);

            } finally {
                if (gr != null) {
                    gr.dispose();
                }
            }
        }

        g2.drawImage(resizedImage, 0, 0, null);

        if (isFPSDisplayed()) {

            String str = String.format("FPS: %.1f", webcam.getFPS());

            int sx = 5;
            int sy = ph - 5;

            g2.setFont(getFont());
            g2.setColor(Color.BLACK);
            g2.drawString(str, sx + 1, sy + 1);
            g2.setColor(Color.WHITE);
            g2.drawString(str, sx, sy);
        }

        if (isImageSizeDisplayed()) {

            String res = String.format("%d\u2A2F%d px", iw, ih);

            FontMetrics metrics = g2.getFontMetrics(getFont());
            int sw = metrics.stringWidth(res);
            int sx = pw - sw - 5;
            int sy = ph - 5;

            g2.setFont(getFont());
            g2.setColor(Color.BLACK);
            g2.drawString(res, sx + 1, sy + 1);
            g2.setColor(Color.WHITE);
            g2.drawString(res, sx, sy);
        }

        if (isDisplayDebugInfo()) {

            if (lastRepaintTime < 0) {
                lastRepaintTime = System.currentTimeMillis();
            } else {

                long now = System.currentTimeMillis();
                String res = String.format("DEBUG: repaints per second: %.1f", (double) 1000 / (now - lastRepaintTime));
                lastRepaintTime = now;
                g2.setFont(getFont());
                g2.setColor(Color.BLACK);
                g2.drawString(res, 6, 16);
                g2.setColor(Color.WHITE);
                g2.drawString(res, 5, 15);
            }
        }

        g2.setRenderingHint(KEY_ANTIALIASING, antialiasing);
        g2.setRenderingHint(KEY_RENDERING, rendering);
    }

I have tried quite a lot of things to optimal my rendering, but there's nothing work. The things I tried:

1. Create compatiable buffered image to render => already done in the code as you can see.
2. Use DoubleBuffer strategy => as I read, this technique is already done in paintComponent() method. I also tried to implement it using the code in the answer here , but also no result.
3. Turn on, off OpenGL, force DirectDraw using VM parameters. No results.

I'm considering to change to an OpenGL library to render the image, but it will be the last option, because I've no knowledge about OpenGL, and I think Java2D is more than enough for my application. Can anyone help me solve this problem?

Answer 1

After test with various things, include render webcam's images using OpenGL (took me too much time since I have to learned OpenGL from scratch), I have found the solution. These are the things I did:

1. Using OpenImja driver instead of default driver. I built OpenImja using Maven, after that I took 4 jar files to put to library path: core-1.1.jar ; core-image-1.1.jar ; core-video-1.1.jar ; core-video-capture-1.1.jar .
2. Scale is no go. In the previous version, I used the best resolution of webcam and scaled it to fit screen size. Big mistake.
3. Force the app to render using DirectDraw with -Dsun.java2d.noddraw=false . According to Oracle, Swing use a mix of DirectDraw and GDI pipeline to render. But since I wrote the app for Window, it is better to only use DirectDraw. For other platforms like Linux, maybe X11 or OpenGL will be better.

After that, my CPU percentage for the app dropped from 33% to 11%. Hope this will have someone who runs to the same problem with me.