Need Help Capturing Global Keyboard Events in UWP

Question

EDIT 4: It seems this only happens when running the app through the debugger. So this is not a major problem.

I have a lot of custom keyboard controls, and a lot of it needs to fire regardless of what control has focus. So I'm using the following code in my MainPage constructor:

public MainPage()
{
    Window.Current.CoreWindow.KeyDown += CoreWindow_KeyDown;
    Window.Current.CoreWindow.KeyUp += CoreWindow_KeyUp;
}

public void public async void CoreWindow_KeyDown(CoreWindow sender, KeyEventArgs args)
{
    // Handle here
}

But I'm having the worst kind of problem. Whenever things are happening on the UI thread it seems to be badly interfering with keyboard input. It's as if it stores some kind of backlog.

For example if I go to an AutoSuggestBox which does a lot of logic each keypress and populates the results with graphics loaded from a server and such, and I type 'abcd' it's quite often the 'd' won't register. Then when I type 'e' a few seconds later, the 'd' will go through but not the 'e'. It's not hardware related, it only does this in my UWP app I'm working on.

In the debugger I have confirmed that this unwanted behavior is all happening before the event is fired. args.VirtualKey is firing a 'd' when I type 'e'.

Also the keyup events only fire like 95% of the time. CoreWindow.PointerWheelChanged does not exhibit this problem. Gamepad input using the same handler as the keyboard does not have this problem either.

The more activity on the UI thread seems to increase the severity of the problem.

Does anyone know how to remedy this situation? Any kind of workaround or alternative solution, or at least an explanation of what might be happening?

EDIT: I tried setting all 4 options for Window.Current.CoreWindow.Dispatcher.ProcessEvents(), no improvement.

EDIT 2: It seems the fact that I'm capturing CoreWindow.Keydown for global events is a non-sequitur. The problem also happens with any regular KeyDown event on any focused control.

EDIT 3: I believe I realize what's happening and I think it's a bug. My cursory understanding is that UWP keyboard input is sandboxed to prevent keylogger malware or something, so there's some lower-level translation between the raw key input and the VirtualKey that CoreWindow processes. That's fine, but it seems it doesn't work right under certain conditions.

When there is load on the UI thread during rapid keyboard input (like typing) it sometimes does not detect key releases. This is why KeyUp doesn't fire occasionally as I mentioned. This also messes up KeyDown because keys are in a lock state it thinks the key is being held down when in reality it is not. Then when the next key release does register, the CoreWindow dispatcher flushes its queue and the result is that it fires both an event for the previous key input as well as for the new one. So type 'abcd' and 'd doesn't fire. Wait 10 seconds and then press 'e'. Suddenly both 'd' and 'e' will appear. Or more likly press 'd' again because it didn't register the first time, and double 'dd' will display. Absolutely unacceptable behavior by any standard.

The best way you can try to reproduce it yourself is use an AutoSuggestBox that does something blocking like queries and loads image thumbnails in the results as you type. And keep in mind even a tiny bit of UI load seems to cause it. Even if I asynchronously preload the image as a stream or byte array, it still blocks the UI thread for 1-2ms when the BitmapImage source is set. 1-2ms less than a frame of video and thus visually imperceptible, but it seems it's enough to occasionally not detect when a keyboard key is released.

This could be something hardware specific. I've tested different keyboards but not a different computer.

Answer 1

You can try this:

public MainPage()
{
    InitializeComponent();
    Window.Current.CoreWindow.CharacterReceived += CoreWindow_CharacterReceived;
}

private async void CoreWindow_CharacterReceived(Windows.UI.Core.CoreWindow sender, Windows.UI.Core.CharacterReceivedEventArgs args)
{
    if (args.KeyCode == 27 ) //Escape
    {
         // your code here fore Escape key
    }
    if (args.KeyCode == 13) //Enter
    {
         // your code here fore Enter key
    }
}

You can use key codes for other keyboard characters.

Answer 2

Activity on the UI thread during rapid keyboard input prevents CoreWindow from detecting a key release, at least on some hardware, targeting 1803 Spring Creators Update. Hopefully someone knows a better solution but for now here is a partial workaround solution:

public void CoreWindow_KeyDown(Windows.UI.Core.CoreWindow sender, Windows.UI.Core.KeyEventArgs args)
{
    CoreVirtualKeyStates keyState = sender.GetAsyncKeyState(args.VirtualKey);
    if ((int)keyState == 3)
    {
        // KeyState is locked and pressed.
        // Whenever this happens it means the event fired when I actaually pressed this key.
        // Handle event.
    }
    else if ((int)keyState == 2)
    {
        // KeyState is locked but not pressed. How if it's not caps lock?
        // When this happens it's an unwanted duplicate of the last keystroke.
        // Do not handle event.
    }
    else if ((int)keyState == 0)
    {
        // Key state is None?!? How can a key that isn't currently down fire a KeyDown event?
        // This is a phantom delayed rection of a missed event from two keystrokes ago.
        // Do not handle event.
    }
}

This will prevent the delayed reaction problem but there will still be missed keystrokes as a result. Not ideal but a vast improvement.

Here is the enum: https://docs.microsoft.com/en-us/uwp/api/windows.ui.core.corevirtualkeystates

Oddly state 3 ("Pressed | Locked") that always represents a correct key input isn't in the documentation.

Also note that 'CoreVirtualKeyStates == 0' makes it clear this is a bug, at least with my hardware. How could a key with the 'None' state have fired a KeyDown event? Nobody's fingers are that fast. I think this is the CoreWindow dispatcher flushing its queue because it missed a KeyUp event.