使用相机快速进行实时人脸跟踪4

Question

I want to be able to track a users face from the camera feed. I have looked at this SO post. I used the code given in the answer but it did not seem to do anything. I have heard that

func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection!)

has been changed to something else in swift 4. Could this be the problem with the code?

While face tracking I want to also monitor face landmarks with CIFaceFeature. How would I do this?

Answer 1

I have found a starting point here: https://github.com/jeffreybergier/Blog-Getting-Started-with-Vision .

Basically you can instatiate a video capture session declaring a lazy variable like this:

private lazy var captureSession: AVCaptureSession = {
    let session = AVCaptureSession()
    session.sessionPreset = AVCaptureSession.Preset.photo
    guard
        let frontCamera = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .front),
        let input = try? AVCaptureDeviceInput(device: frontCamera)
        else { return session }
    session.addInput(input)
    return session
}()

Then inside viewDidLoad you start the session

self.captureSession.startRunning()

And finally you can perform your requests inside

func captureOutput(_ output: AVCaptureOutput, 
    didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
}

for example:

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: 
    CMSampleBuffer, from connection: AVCaptureConnection) {
    guard
        // make sure the pixel buffer can be converted
        let pixelBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
        else { return }

    let faceRequest = VNDetectFaceRectanglesRequest(completionHandler: self.faceDetectedRequestUpdate)

    // perform the request
    do {
        try self.visionSequenceHandler.perform([faceRequest], on: pixelBuffer)
    } catch {
        print("Throws: \(error)")
    }
}

And then you define your faceDetectedRequestUpdate function.

Anyway I have to say that I haven't been able to figure out how to create a working example from here. The best working example I have found is in Apple's documentation: https://developer.apple.com/documentation/vision/tracking_the_user_s_face_in_real_time