Hand Tracking in MSFS - Leap Motion / UltraLeap native support for WMR/OpenXR

@RomanDesign, since I posted my explanation above, I’ve been prototyping the “hand tracking to controller” emulation OpenXR API layer, and after a couple of Saturday afternoons, I’ve reached some pretty encouraging results. Using a Leap Motion Controller and a Windows Mixed Reality headset (Acer in my case, but it will work with Reverb too), I am able to use my hands to grab the yoke and navigate the buttons on the control panel (like the video on the thread).

There’s a lot of tweaking/configuration to do to make it a pleasant experience (hand offset and gesture calibration), and also since I am not an MSFS aficionado, I’m not sure what a pleasant experience should be like :slight_smile:

I’ll try to make a video soon, and after a bit more testing, I’ll release the project on GitHub (it’s under 1,000 lines of C++ so far). I’ll be looking for people to test it too.

Note that this is a personal project and not affiliated with Ultraleap nor Microsoft.

9 Likes