How can I recognise HoloLens gestures in C#?

Question

So the last days I searched the Internet on how to recognise HoloLens gestures in C#/Unity (for instance a normal Tap Gesture).

I for instance tried the GestureRecognizer but it seems to be outdated and I was not able to get it to work.

I tested it by emulating in Unity(Mixed Reality Toolkit) and holding 'Space' in order to see the hand and clicked to left mouse button.

That way I can interact with objects like using the HoloLens normally, but I was not able to get the GestureRecognizer to work.

Any code snippets would be helpful. I am simply trying to Log (or later on call a method) if a single tap or double tap was recognised.

Answer 1

I think there are a few pieces at play here. First, the Mixed Reality Toolkit's simulated hands don't send their events through the regular GestureRecognizer, which would explain why you aren't seeing events there. They do send input and gesture events through the MRTK's input system though. You can listen to those events via something like:

public class TapListener : MonoBehaviour, IMixedRealityGestureHandler
{
    [SerializeField]
    private MixedRealityInputAction selectAction; // You'll need to set this in the Inspector to Select

    private void OnEnable()
    {
        CoreServices.InputSystem?.RegisterHandler<IMixedRealityGestureHandler>(this);
    }

    private void OnDisable()
    {
        CoreService.InputSystem?.UnregisterHandler<IMixedRealityGestureHandler>(this);
    }

    public void OnGestureCompleted(InputEventData eventData)
    {
        if (eventData.MixedRealityInputAction == selectAction)
        {
            Debug.Log("Tap!");
        }
    }

    public void OnGestureStarted(InputEventData eventData) { }
    public void OnGestureUpdated(InputEventData eventData) { }
    public void OnGestureCanceled(InputEventData eventData) { }
}

Second, the MRTK doesn't currently support double tap, but it'd be great if you could file a feature request on the repo so it can be logged.