It’s a 3090, the latest .17 beta isn’t helping me, .16 was super smooth, best experience I’ve had in the sim.
1 Like
Nvidia 3090 RTX
1 Like
OK, I finally have a video that demonstrated the hand tracking operation, including the interaction loss problem. It’s part of my large video on my Motion VR Cockpit, but this link should start at the right time for hand tracking demo. You can watch how it works and also see an interaction loss bug.
As you can see, in general the hand tracking works well. It’s precise enought for me to hit all kinds of buttons quickly without much effort, even a tiny Flight Director switch. The click works on the first try about 90% of the time. So it’s very usable, but two issues are preventing it from being practical and need to be fixed or worked around. @mbucchia please please fix this. Hand tracking is 98% there, it just need a tiny little push to be ready for prime time!
-
The ugly interaction loss bug! It looks like a product of bad MSFS VR controller implementation, exacerbated by OXRToolkit. Looks like it happens every time a hand goes out of camera view, but it can also happen when hand is tracked at all times, I tested that. It has to be fixed, or if the fix is not possible, at least an ugly workaround could work: just make the controller present when hand is outside of camera view, somewhere out of view or at the bottom of the view, and activate the “squeeze” button every 10 seconds or so, but only while controller is not moving (not active). It should be an easy thing to implement. This would keep the virtual controller active in MSFS and it won’t be needed to wake up every time, which is annoying even with hardware VR Controller, but is a deal-breaker with a hand. Additionally, it’s better to only emulate one hand, as MSFS can only operate one controller at a time (why???) so one hand will lose interaction when another one is activated.
-
“finger gun” button push doesn’t work! It’s there in the config and it’s supposed to work, but it doesn’t. Because hand tracking is less precise than hardware VR controller tracking, pushing small buttons, like Boeing FMC or GPS takes more effort and precision. It’s possible but not as fast as with a controller. It would be much faster if you could put your hand in a “finger gun” gesture and just push buttons (so behind the scenes it would only trigger the button right after contact - when a control turns blue). MSFS didn’t do it this way, so it has to be faked. This would allow quick button pushing.
There is also the issue of “laser” mode being hard to aim because of less precision, but that one is not fixable.
- A wish: as the majority of flight simmers would only use one hand (usually right) to operate switches while another hand is on the joystick, maybe @mbucchia you could add an option for OXRToolkit to only override one VR controller, while keep a second hardware controller active? That way we could use VR controller in laser mode for menus, or use it for a lot of FMC programming if it’s faster than using a hand, but then we could use a hand in flight, where it’s needed the most. Controller get in the way of handling throttes etc. but it’s OK to use on the ground when setting up the flight. So if it’s possible it would be nice. As it is now, OXRToolkit hands can’t coniside with hardware controllers. But maybe if you only use one and allow the other to be hardware, they can?
Note: you can’t see the actual cubist hand representation in the video because I captured the main display. It’s there in the HMD. But you can see the VR “pointer”.
2 Likes
Hey RomanDesign, thanks for the great video and information. I was curious if you could share screenshots of your OpenXR Toolkit Hand Configuration settings? In particular, I am interested in how much you adjusted the tolerances and what gestures you’re using. I was pushing hard on this front a month or two ago but couldn’t find a way to reliably engage switches without having the pointer still highlighting the control by the time the gesture was made. In the video, its very difficult to determine what gesture you are using because its the view of the back of your hand. You seem to be able to point and have the click be unrelated to the pointing finger. If you could share your setup, I’d be very much interested in testing it with my Leap Motion device and my VR setup.
Cheers!
FauxFlights
I have been having a great time with the beta using DX11 and DLSS, but seeing the X-Plane12 stuff had me wanting to revisit using the OpenXR Toolkit to leverage the color adjustments for MSFS. I reenabled yesterday and was dialing it in successfully which brought me to this thread. It had seemed like there was a lot of success with DX12 and I had read about people doing 200% resolution with Balanced DLSS so today I wanted to give those a try.
For my setup, and I will try to troubleshoot this more later, DX12 with OpenXR Toolkit gives me a CTD. If I disable the OpenXR toolkit, I can launch VR without a CTD using DX12. I can use DX12 and I can use DX11/OpenXR, but not DX12/OpenXR. As soon as I enable VR, hard crash. I will give the prescribed FFR disabling a try but I had thought reading the thread that I should have been okay without it. I will also test the OpenXR Toolkit safe mode. As I said, I’ll push on this as I have time and report back.
My setup is a 3080 with a Valve Index running SteamVR.
There is only mention of DX12.
The current beta build (1.27.18 or 19) has a known issue with DX12 (texture memory corruption/probably GPU driver bug) and Asobo has restricted some features to check the issue.
As a result, DX12 performance on nVidia GPUs is not available.
Asobo has officially announced that both the MSFS software and nVidia GPU drivers will be fixed in the next version, probably the official SU10 version.
2022/09/08 Devupdate blog
Therefore, we do not recommend testing the performance of the DX12+OpenXR combination with the current version. It would be better to wait for a week or so and wait for a newer version.
Of course, there is nothing wrong with testing the OpenXR side.
Oh, thanks so much! Apologies for jumping the gun. I will revert back to DX11 and proceed with the rest of what I was looking to do. Thanks again for the thorough explanation.
1 Like
Sure! I spent quite a bit of time finding the most convenient settings for me. Note that I’m using INDEX TIP for aim, i.e. my index finger determines where controller is pointing, which is more twitchy but much more natural than trying to aim with the whole palm. So Translation and Orientation settings are based on that: aligning the controller arrow with the tip of my index finger.
BTW I’m only using right hand, to avoid interaction loss because MSFS can only use one controller at a time - which doesn’t make any sense and is a bad Asobo VR controller implementation, but that’s what we have.
I assigned “pinch” to menu open. “Thumb press” switches between laser/direct mode, which I have to do often to reenable interaction - until @mbucchia can implement the “keep-alive” workaround or another fix for the interaction loss. “Finger gun” (touching your thumb to the squeezed middle finger while pointing forward with the index finger) is used for trigger press - I found by experimentation that this is the easiest gesture to do while pointing your index finger without losing aim. All others result in the aim shifting from the click spot. “Squeeze” is used for squeeze (default).
Gesture settings:
Haptics - this doesn’t work. It was originally my idea to implement this - it’s supposed to trigger the trigger press when you approach any button with your index finger in a “finger gun” position with engaged trigger. On other words, instead of normal “touch until it turns blue, then trigger with thumb on middle finger” it would be “touch thum on middle finger in a finger gun gesture, and in that position just puch buttons - they would trigger automatically as soon as the turn blue”. If you hand is in a “finger gun” gesture already when it reaches interaction spot (haptic click/buzz is sent by MSFS) - OpenXR Toolkit emulates a trigger press. iBut it doesn’t work for me at all.
1 Like
Thanks so much for taking the time to put this together. Reading your description, I’m laughing at myself because I think I was trying to do finger-gun by using my middle-finger as the trigger not realizing it was the thumb. Obviously its hard to keep that pointing finger on the target while pulling your middle finger to trigger. Wiggling my thumb is going to be much easier. lol
Thanks again. This is super helpful and I’ll be sure to share back anything I find that is of use to the effort.
1 Like
Are you still having the issue with latest 112.2209.2003? If so, please join the conversation on the debugging thread:
Thanks.
If you are talking about the OpenXR Developer Tool “latest runtime” preview - I have an exactly the same problem you are describing, freshly updated runtime which was the only change between flights with reboot, see my postst in the topic @mbucchia referred to. OpenXR Toolkit menu judders badly, just like you are describing, and I’m having huge motion reprojection warping artifacts even when stable locked 30FPS MR, which didn’t happen before the update, when everything was incredibly smooth.
There is a posting up sways that gives instructions on how to use OpenXr Toolkit with dX12. It’s referencing a reg key - pretty easily fix. Basically disables Foveated Rendering which doesn’t work with DX12
Hello @mbucchia ,
I have a question about next steps with new version, so SU10 (btw. I dont use beta). Looks like in this week we will recive SU10 as official update. Im using OpenXR Toolkit - v1.1.3, so ok, when the new update will be released, what should I do to use OpenXR Toolkit still (I must to remove the old folder, unistall something, install etc). Can you give some info about it? Thank you in advance. 
I don’t really have a clear date, but hoping within 2 weeks. This means there might be a short period of time between official SU10 release and official OpenXR Toolkit release. I don’t want to release OpenXR Toolkit update at the same exact time than SU10 because this might make it difficult to distinguish SU10 vs OpenXR Toolkit issues. I unfortunately seem to have missed the window to release before SU10.
In theory, releasing OpenXR Toolkit a week or so after SU10 shouldn’t affect too many of you negatively: the public version of OpenXR Toolkit works with SU10 as long as you don’t use DX12 and/or eye-tracked foveated rendering. It’s only a few days so I hope people can be patient.
Trying to cram as many things as possible into the next version, because it will likely be the last version of OpenXR Toolkit for a while (minus a quick patch release to address any major blocker).
10 Likes
It would be greatly appreciated if you could squeeze in a very simple “keep-alive” feature for hand tracking. Something that wouldn’t take much of your time at all (hopefully): at a minimum, just an option in “mapping tool” app that enables triggering “squeeze” every X seconds. If it doesn’t complicate things, only when the actual hand is not in the frame / not moving. Just a reminder. I know you’re pressed on time, I understand if you can’t, but if it’s a quick thing to do - it would be a welcome addition. Hopefully this would make hand tracking fully usable. Other hand tracking improvements can come later, but this constatnt interaction loss problem, which this workaround hopefully would cure, is what makes it not practical yet.
I haven’t had time to read all your details in previous messages, but it sounded like an MSFS bug. Has anynoby reported it to them?
Thank you, so for me, if I stay with DX11 the best option is do not turn on DLSS till the next update of your tool? I use sharpness option so I must use FSR.
If you are on DX11 then you should not have any issues. And you can try DLSS, there shouldn’t be any impact I believe.
2 Likes
Is it both tracked and fixed foveated rendering we won’t be able to use with SU10 final release on DX12?.. If so I’ll go back to DX11 straight away to get the benefit’s of FFR back.
Since I’m on possibly the only headset that doesn’t currently work at all with SU10 beta & OpenXR toolkit (Vive Pro 2) I’m certainly eagerly waiting for the update. Just doing DLSS for now but I imagine the significant benefits will be from DLSS + FSR.