如何在 JavaFX 應用程序中顯示 OpenCV 網絡攝像頭捕獲幀

Question

我正在使用JavaFX創建桌面應用程序，它允許您從網絡攝像頭掃描二維碼。

我決定選擇 JavaCV 來處理網絡攝像頭捕獲。 但是，問題在於CanvasFrame class 創建了 Swing JFrame 。 我的主要目標是找到將其與 JavaFX 組件集成的最佳方法。

我的問題是是否可以在JPanel （或其他 Swing/JavaFx 組件）中創建CanvasFrame ，而不是在JFrame中創建。 在這個選項中，我會將JPanel包裝到SwingNode - 它解決了我的集成問題。

在我的情況下，我還要求其他解決 JavaFX 與 JavaCV 集成問題的建議。 也許有一種直接的方法可以將攝像頭屏幕嵌入到 JavaFx 組件中。

我在下面粘貼測試代碼。 我的代碼是用kotlin寫的，但是不影響問題：

import com.google.zxing.*
import com.google.zxing.client.j2se.BufferedImageLuminanceSource
import com.google.zxing.common.HybridBinarizer
import org.bytedeco.javacv.*
import java.awt.image.BufferedImage
import java.util.*
import java.util.concurrent.Executors

class Test {
    companion object {
        @JvmStatic
        fun main(args: Array<String>) {
            Executors.newSingleThreadExecutor().execute { testWebcam() }
        }

        private fun testWebcam() {
            val grabber: OpenCVFrameGrabber = OpenCVFrameGrabber(0);
            val canvasFrame: CanvasFrame = CanvasFrame("Cam")
            grabber.start()

            while (canvasFrame.isVisible) {
                val frame: Frame = grabber.grabFrame() ?: break
                canvasFrame.showImage(frame)
                decodeQrCode(grabber)
            }
        }

        private fun decodeQrCode(grabber: OpenCVFrameGrabber) {
            val java2DFrameConverter: Java2DFrameConverter = Java2DFrameConverter()

            val frame: Frame = grabber.grabFrame()
            val image = java2DFrameConverter.getBufferedImage(frame)

            val decodedQr = parseQr(image)
            println(decodedQr)
        }

        private fun parseQr(image: BufferedImage): String? {

            val reader: MultiFormatReader = MultiFormatReader()
            val binaryBitmap: BinaryBitmap =
                BinaryBitmap(HybridBinarizer(BufferedImageLuminanceSource(image)))

            val hints: Hashtable<DecodeHintType, Any> = Hashtable()
            hints[DecodeHintType.CHARACTER_SET] = "UTF-8"
            hints[DecodeHintType.POSSIBLE_FORMATS] = listOf(BarcodeFormat.QR_CODE)

            return try {
                reader.decode(binaryBitmap, hints).text
            } catch (e: NotFoundException) {
                null;
            }

        }
    }
}

Answer 1

There's a project at https://github.com/rladstaetter/javacv-webcam that has examples of using javacv with Swing, JavaFX, and a newer method of using a shared memory buffer between OpenCV and JavaFX's PixelBuffer.

您可以使用由共享 ByteBuffer 支持的 JavaFX 的 ImageView，而不是使用 CanvasFrame。 偽代碼是：

import java.nio.ByteBuffer

import javafx.scene.image._
import org.bytedeco.javacv.Frame
import org.bytedeco.opencv.global.opencv_imgproc._
import org.bytedeco.opencv.opencv_core.Mat

val videoView: ImageView = ImageView()
val grabber: OpenCVFrameGrabber = OpenCVFrameGrabber(0)
grabber.start()

// Fire off a thread to grab frames while the camera is active
// Each frame will ber passed to the updateView method below
// ... timer/thread omitted for brevity

val javaCVMat = Mat()

/** create buffer only once saves much time! */
val buffer: ByteBuffer = javaCVMat.createBuffer()

val formatByte: WritablePixelFormat<ByteBuffer> = PixelFormat.getByteBgraPreInstance()

fun updateView(frame: Frame): Unit = {
  val w = frame.imageWidth()
  val h = frame.imageHeight()

  val mat = javaCVConv.convert(frame)
  cvtColor(mat, javaCVMat, COLOR_BGR2BGRA)

  val pb = PixelBuffer(w, h, buffer, formatByte)
  val wi = WritableImage(pb)
  videoView.setImage(wi)
}

Answer 2

我解決了我的問題。 就我而言，最好的解決方案是使用Java2DFrameConverter ：

import javafx.application.Application
import javafx.embed.swing.SwingFXUtils
import javafx.scene.Scene
import javafx.scene.image.ImageView
import javafx.scene.image.WritableImage
import javafx.scene.layout.VBox
import javafx.stage.Stage
import org.bytedeco.javacv.Frame
import org.bytedeco.javacv.Java2DFrameConverter
import org.bytedeco.javacv.OpenCVFrameGrabber
import java.awt.image.BufferedImage
import java.util.concurrent.Executors

class StackOverflow : Application() {
    private val java2DFrameConverter: Java2DFrameConverter = Java2DFrameConverter()

    companion object {
        @JvmStatic
        fun main(args: Array<String>) {
            launch(StackOverflow::class.java)
        }
    }

    override fun start(primaryStage: Stage) {
        val grabber: OpenCVFrameGrabber = OpenCVFrameGrabber(0)
        grabber.start()

        val imageView: ImageView = ImageView()

        Executors.newSingleThreadExecutor().execute {
            while (true) {
                val frame = grabber.grabFrame()
                imageView.image = frameToImage(frame)
            }
        }

        val scene: Scene = Scene(VBox(imageView), 800.0, 800.0)
        primaryStage.scene = scene
        primaryStage.show()
    }

    private fun frameToImage(frame: Frame): WritableImage {
        val bufferedImage: BufferedImage = java2DFrameConverter.getBufferedImage(frame)
        return SwingFXUtils.toFXImage(bufferedImage, null)
    }
}