are ARFragment and GLSurfaceView compatible together in one Activity?

Question

I'm trying to combine ARCore examples - hellosceneform and augmented_image_java together in my Android app to be a single Activity. ie it will be possible to touch the screen and drop and AR object, and also to for the camera to be scanning for objects it recognises and put an AR Frame for example around the object. Please help?

Answer 1

Sceneform implements its own rendering engine in the SceneView class. This makes it incompatible with GLSurfaceView. You can do what you are looking to do by combining the hellosceneform sample and the augmentedimage sample, which are both using Sceneform.

To merge the two, start with AugmentedImageActivity.java and add the member variable for the model:

private ModelRenderable andyRenderable;

You'll also need the model of Andy.

Then in onCreate, add the loading of the model and handling of the tapping at the end of onCreate() :

   @Override
    protected void onCreate(Bundle savedInstanceState) {
      super.onCreate(savedInstanceState);
      setContentView(R.layout.activity_main);

      arFragment = (ArFragment) 
      getSupportFragmentManager().findFragmentById(R.id.ux_fragment);
      fitToScanView = findViewById(R.id.image_view_fit_to_scan);
   arFragment.getArSceneView().getScene().addOnUpdateListener(this::onUpdateFrame);

    /*** Add HelloSceneform functionality here vvvvvvvvv **/

    // When you build a Renderable, Sceneform loads its resources in
    // the background while returning a CompletableFuture.
    // Call thenAccept(), handle(), or check isDone() before calling get().
    ModelRenderable.builder()
            .setSource(this, R.raw.andy)
            .build()
            .thenAccept(renderable -> andyRenderable = renderable)
            .exceptionally(
                    throwable -> {
                      Toast toast =
                              Toast.makeText(this,
                             "Unable to load andy renderable", Toast.LENGTH_LONG);
                      toast.setGravity(Gravity.CENTER, 0, 0);
                      toast.show();
                      return null;
                    });

    arFragment.setOnTapArPlaneListener(
            (HitResult hitResult, Plane plane, MotionEvent motionEvent) -> {
              if (andyRenderable == null) {
                return;
              }

              // Create the Anchor.
              Anchor anchor = hitResult.createAnchor();
              AnchorNode anchorNode = new AnchorNode(anchor);
              anchorNode.setParent(arFragment.getArSceneView().getScene());

              // Create the transformable andy and add it to the anchor.
              TransformableNode andy = new TransformableNode(arFragment.getTransformationSystem());
              andy.setParent(anchorNode);
              andy.setRenderable(andyRenderable);
              andy.select();
            });
  }

The augmented image fragment turns off the plane discovery, so you'll want to remove that code so planes are rendered. This code is in AugmentedImageFragment.java . Simply remove these lines:

// Delete these lines to keep the plane discovery.
getPlaneDiscoveryController().hide();
getPlaneDiscoveryController().setInstructionView(null);
getArSceneView().getPlaneRenderer().setEnabled(false);