ARKit how to draw face mesh?

Question

I want to draw the face mesh in real time as is shown in the Apple video . It's also being done in the MeasureKit's app too. I got the ARSession running which constantly delivers updated ARFrame objects in delegate and I can get ARFaceAnchor from it which contains the face geomatery consisting of ARFaceGeometry and blendShapes. How to use the ARFaceGeometry data to draw the mesh in real time ?

Thanks.

Answer 1

It's possible to create a face mesh in augmented reality. I'd recommend using the following innovative approach which utilises ARSCNViewDelegate.

For example:

extension ViewController: ARSCNViewDelegate {

  func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
    guard let device = sceneView.device else {
      return nil
    }
    let faceGeometry = ARSCNFaceGeometry(device: device)
    let node = SCNNode(geometry: faceGeometry)
    node.geometry?.firstMaterial?.fillMode = .lines
    return node
  }
}

In this example, we created a face geometry to be rendered. Next, we need a SceneKit node based on that face geometry. We require the fill mode for the node's material to be fine lines. This should achieve the face mesh that you're after. A further approach to this would be for the face mesh to react to facial expressions.