SceneKit unprojectPoint() with custom camera

Question

I am developing a molecular visualizer for macOS / iPadOS with SceneKit. Long story short, I want that when the user clicks (or touches) the screen at a certain position, a new atom is placed (in this example just a SCNSphere).

Previously, I had the allowsCameraControl property of the SCNView active, which allowed me to freely move the camera and with the unprojectPoint() method, I could successfully place a new node at touch location. The limitation of the default camera controller is that it does not zoom. When you pinch the screen, it changes the FOV property of the camera instead of moving it through the Z axis.

Therefore, I made a custom camera node with a SCNCamera. I succesfully recreated the default camera behaviour (movement, rotation) and furthermore I am able to correclty zoom into the scene. The downside of this is that the unprojectPoint() method no longer works as expeced, as the new nodes are placed at a very close position of the camera node itself. No matter where I click on the scene, that the unprojected point will always be very close to 0, 0, 10

internal func newNodeAt(point: CGPoint) {
        let pointVector = SCNVector3(point.x, point.y, 0.8)
        let position = self.unprojectPoint(pointVector)
        
        print("x:\(position.x), y: \(position.y), z: \(position.z)")
        
        let newSphere = SCNSphere(radius: 1)
        let newNode = SCNNode(geometry: newSphere)
        
        self.scene?.rootNode.addChildNode(newNode) 
}

The camera node is setup as folows and its directly attached to the scene root node.

    internal func setupCameraNode() -> SCNNode {
        let cam = SCNCamera()
        cam.name = "camera"
        cam.zFar = 200
        cam.zNear = 0.1
        let camNode = SCNNode()
        camNode.camera = cam
        camNode.position = SCNVector3(0, 0, 5)
        camNode.name = "Camera node"
        return camNode
    }

These are the printed positions after clicking on random positions of the scene.

x:-0.1988764852285385,  y: -0.05589345842599869, z: 10.920427322387695
x:-0.18989555537700653, y:  0.14564114809036255, z: 10.920427322387695
x: 0.2168566882610321,  y:  0.13085339963436127, z: 10.920427322387695
x: 0.24202580749988556, y: -0.15493911504745483, z: 10.920427322387695
x:-0.06516486406326294, y: -0.1781780868768692,  z: 10.920427322387695
x:-0.08134553581476212, y:  0.12478446960449219, z: 10.920427322387695
x:-0.25866374373435974, y:  0.1456427276134491,  z: 10.920427322387695
x: 0.217658132314682,   y:  0.16270162165164948, z: 10.920427322387695
x: 0.2053154855966568,  y: -0.12679903209209442, z: 10.920427322387695

I suppose that the unprojectPoint() is somehow related to the point of view? But I do not know how to fix this. Thanks.

Answer 1

I think you are on the right track, you just have to provide some kind of depth reference for the user. This is my code for similar, but when I call airStrike, I deal with the depth based on a plane facing the user and that's how I know where Z needs to be.

Just a guess without a visual, but seems there are a couple of options. Create a reference plane in the middle of the molecule and ++/-- that to show where the tap will land from a depth perpective.

Or just let them put it anywhere, then select it and depth++/depth-- to get it in the right position.

@objc func handleTap(recognizer: UITapGestureRecognizer)
{
 let location: CGPoint = recognizer.location(in: gameScene)
            
 if(data.isAirStrikeModeOn == true)
 {
  let projectedPoint = gameScene.projectPoint(SCNVector3(0, 0, 0))
  let scenePoint = gameScene.unprojectPoint(SCNVector3(location.x, location.y, CGFloat(projectedPoint.z)))
  gameControl.airStrike(position: scenePoint)
 }
}

Answer 2

After days of testing I figured out a workaround and now I can place the nodes correctly where they should be.

My node tree was is like this:

• RootNode
  • CameraNode
  • atomNodes
    • atom (individual spheres)

Therefore, all I had to do was to convert the unprojected position from the RootNode (which I suppose is the one that the camera takes the reference from) to the atomNodes, thus:

let unprojected = unprojectPoint(SCNVector3(location.x, location.y, 0.99))                                     
let position = atomNodes.convertPosition(unprojected, from: rootNode)

The 0.99 is just a nice Z position in my view for the spheres to be placed. (More info here )

My advice would be to always check the node tree because the positions are relative to each other.