如何让实体不断查看 RealityKit 中的相机，类似于 SceneKit 中的 Billboard Constraint

Question

I have an app that I am trying to update from SceneKit to RealityKit, and one of the features that I am having a hard time replicating in RealityKit is making an entity constantly look at the camera. In SceneKit, this was accomplished by adding the following billboard constraints to the node:

let billboardConstraint = SCNBillboardConstraint()

billboardConstraint.freeAxes = [.X, .Y]
startLabelNode.constraints = [billboardConstraint]

Which would allow the startLabelNode to freely rotate so that it was constantly facing the camera without the startLabelNode changing its position.

However, I can't seem to figure out a way to do this with RealityKit. I have tried the "lookat" method, which doesn't seem to offer the ability to constantly face the camera. Here is a short sample app where I have tried to implement a version of this in RealityKit, but it doesn't offer the ability to have the entity constantly face the camera like it did in SceneKit:

import UIKit
import RealityKit
import ARKit

class ViewController: UIViewController, ARSessionDelegate {
    @IBOutlet weak var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        arView.session.delegate = self
        arView.environment.sceneUnderstanding.options = []
        arView.debugOptions.insert(.showSceneUnderstanding) // Display a debug visualization of the mesh.
        arView.renderOptions = [.disablePersonOcclusion, .disableDepthOfField, .disableMotionBlur] // For performance, disable render options that are not required for this app.
        arView.automaticallyConfigureSession = false
        let configuration = ARWorldTrackingConfiguration()
        if ARWorldTrackingConfiguration.supportsSceneReconstruction(.mesh) {
            configuration.sceneReconstruction = .mesh
        } else {
            print("Mesh Classification not available on this device")
            configuration.worldAlignment = .gravity
            configuration.planeDetection = [.horizontal, .vertical]
        }
        configuration.environmentTexturing = .automatic
        arView.session.run(configuration)
        
        UIApplication.shared.isIdleTimerDisabled = true // Prevent the screen from being dimmed to avoid interrupting the AR experience.
    }

    @IBAction func buttonPressed(_ sender: Any) {
        let screenWidth = arView.bounds.width
        let screenHeight = arView.bounds.height
        let centerOfScreen = CGPoint(x: (screenWidth / 2), y: (screenHeight / 2))
        
        if let raycastResult = arView.raycast(from: centerOfScreen, allowing: .estimatedPlane, alignment: .any).first
        {
            addStartLabel(at: raycastResult.worldTransform)
        }
    }
    
    func addStartLabel(at result: simd_float4x4) {
        let resultAnchor = AnchorEntity(world: result)
        resultAnchor.addChild(clickToStartLabel())
        arView.scene.addAnchor(resultAnchor)
    }
    
    func clickToStartLabel() -> ModelEntity {
        let text = "Click to Start Here"
        let textMesh = MeshResource.generateText(text, extrusionDepth: 0.001, font: UIFont.boldSystemFont(ofSize: 0.01))
        let textMaterial = UnlitMaterial(color: .black)
        let textModelEntity = ModelEntity(mesh: textMesh, materials: [textMaterial])
        textModelEntity.generateCollisionShapes(recursive: true)
        textModelEntity.position.x -= textMesh.width / 2
        textModelEntity.position.y -= textMesh.height / 2
        
        let planeMesh = MeshResource.generatePlane(width: (textMesh.width + 0.01), height: (textMesh.height + 0.01))
        let planeMaterial = UnlitMaterial(color: .white)
        let planeModelEntity = ModelEntity(mesh: planeMesh, materials: [planeMaterial])
        planeModelEntity.generateCollisionShapes(recursive:true)
        // move the plane up to make it sit on the anchor instead of in the middle of the anchor
        planeModelEntity.position.y += planeMesh.height / 2
        planeModelEntity.addChild(textModelEntity)
        
        // This does not always keep the planeModelEntity facing the camera
        planeModelEntity.look(at: arView.cameraTransform.translation, from: planeModelEntity.position, relativeTo: nil)
                
        return planeModelEntity
    }
}

extension MeshResource {
    var width: Float
    {
        return (bounds.max.x - bounds.min.x)
    }

    var height: Float
    {
        return (bounds.max.y - bounds.min.y)
    }
}

Is the lookat function the best way to get the missing feature working in RealityKit or is there a better way to have a Entity constantly face the camera?

Answer 1

k - I haven't messed with RK much, but assuming entity is a scenekit node? - then set constraints on it and it will be forced to face 'targetNode' at all times. Provide that works the way you want it to, then you may have to experiment with how the node is initially created IE what direction it is facing.

func setTarget()
    {
        node.constraints = []
        let vConstraint = SCNLookAtConstraint(target: targetNode)
        vConstraint.isGimbalLockEnabled = true
        node.constraints = [vConstraint]
    }

Answer 2

I was able to figure out an answer to my question. Adding the following block of code allowed the entity to constantly look at the camera:

func session(_ session: ARSession, didUpdate frame: ARFrame) {
    planeModelEntity.look(at: arView.cameraTransform.translation, from: planeModelEntity.position(relativeTo: nil), relativeTo: nil)
}