ARKit: How can I add a UIView to ARKit Scene?

Question

I am working on an AR project using ARKit.

I want to add a UIView to ARKit Scene. When I tap on an object, I want to get information as a "pop-up" next to the object. This information is in a UIView.

Is it possible to add this UIView to ARKit Scene? I set up this UIView as a scene and what can I do then? Can I give it a node and then add it to the ARKit Scene? If so, how it works?

Or is there another way?

Thank you!

EDIT: Code of my SecondViewController

class InformationViewController: UIViewController {

    @IBOutlet weak var secondView: UIView!

    override func viewDidLoad() {
        super.viewDidLoad()
        self.view = secondView
    }
}

EDIT 2: Code in firstViewController

guard let secondViewController = storyboard?.instantiateViewController(withIdentifier: "SecondViewController") as? SecondViewController else {
    print ("No secondController")
    return
}

let plane = SCNPlane(width: CGFloat(0.1), height: CGFloat(0.1))       
plane.firstMaterial?.diffuse.contents = secondViewController.view

let node = SCNNode(geometry: plane)

I only get a white screen of a plane, not the view.

Answer 1

The simplest (although undocumented) way to achieve that is to set a UIView backed by a view controller as diffuse contents of a material on a SCNPlane (or any other geometry really, but it works best with planes for obvious reasons).

let plane = SCNPlane()
plane.firstMaterial?.diffuse.contents = someViewController.view
let planeNode = SCNNode(geometry: plane)

You will have to persist the view controller somewhere otherwise it's going to be released and plane will not be visible. Using just a UIView without any UIViewController will throw an error.

The best thing about it is that it keeps all of the gestures and practically works just as a simple view. For example, if you use UITableViewController 's view you will be able to scroll it right inside a scene.

I haven't tested it on iOS 10 and lower, but it's been working on iOS 11 so far. Works both in plain SceneKit scenes and with ARKit.

Answer 2

I cannot provide you code now but this is how to do it.

1. Create a SCNPlane .
2. Create your UIView with all elements you need.
3. Create image context from UIView .
4. Use this image as material for SCNPlane .

Or even easier make SKScene with label and add it as material for SCNPlane .

Answer 3

To place text in a label in the world you draw it into an image and then attach that image to a SCNNode .

For example:

let text = "Hello, Stack Overflow."
let font = UIFont(name: "Arial", size: CGFloat(size))
let width = 128
let height = 128

let fontAttrs: [NSAttributedStringKey: Any] = 
[NSAttributedStringKey.font: font as UIFont]

let stringSize = self.text.size(withAttributes: fontAttrs)
let rect = CGRect(x: CGFloat((width / 2.0) - (stringSize.width/2.0)),
                  y: CGFloat((height / 2.0) - (stringSize.height/2.0)),
                  width: CGFloat(stringSize.width),
                  height: CGFloat(stringSize.height))

let renderer = UIGraphicsImageRenderer(size: CGSize(width: CGFloat(width), height: CGFloat(height)))
let image = renderer.image { context in

    let color = UIColor.blue.withAlphaComponent(CGFloat(0.5))

    color.setFill()
    context.fill(rect)

    text.draw(with: rect, options: .usesLineFragmentOrigin, attributes: fontAttrs, context: nil)
}

let plane = SCNPlane(width: CGFloat(0.1), height: CGFloat(0.1))
plane.firstMaterial?.diffuse.contents = image

let node = SCNNode(geometry: plane)

EDIT: I added these lines:

let color = UIColor.blue.withAlphaComponent(CGFloat(0.5))

color.setFill()
context.fill(rect)

This lets you set the background color and the opacity. There are other ways of doing this - which also let you draw complex shapes - but this is the easiest for basic color.

EDIT 2: Added reference to stringSize and rect