OpenXR Toolkit (upscaling, world scale, hand tracking...) - Release thread

Folks I hear you on the motion reprojection stuff.

Honestly, there’s nothing more I want than improving the WMR side of things, but it’s really difficult to find time for these, especially because it’s really difficult to measure the improvements.

Plug in a third party motion reprojection algorithm

This is some really hard stuff :slight_smile: What you are talking about (a way to externally provide your own algorithm) is something I have prototyped and wanted to open-source, but I am already not able to find developers interested in working on OpenXR Toolkit :smiley: so something like motion reprojection, which is like 100x more complex… This just wasn’t a solution that is sustainable for me long term. Plus I am technically tainted since I have access to MS intellectual property, and I would not be able to (ethically) reuse it.

Does anyone know if the team who work on MR can take feature requests

That’s me. This is a more efficient approach than me doing my own thing to cover all devices out there. I can start from the existing WMR implementation and improve it for WMR only. I have started several efforts in the WMR team to make improvements to our OpenXR implementation. It took several weeks of on/off work and learning on the side, but I am now pretty knowledgeable about the process of motion reprojection (I wrote a high-level post about it on the forum too, but think that the amount of knowledge/complexity under the hood is much much much more things I have not even shared).

It’s going to take a long time to get those out the door, but I already have 2 major changes, one to reduce the gap between our OpenXR implementation and our SteamVR implementation, and another one to take advantage of Nvidia Optical Flow (on RTX cards only). These are currently pending code review internally, so hopefully they’re not too far away (but don’t ask me for timeline!). They both aim at improving the quality of motion estimation (aka motion vectors) which effectively reduces artifacts and wobbling.

Some samples of what improvements look like. See captions. Taken without prop-mod. Frame rate was 1/3rd (30 FPS).

First half of the video: what you have today; Second half: improvement #1

First half of the video: what you have today; Second half: improvement #2 (Nvidia RTX only) combined with #1

These are the 2 “most successful” improvements out of a dozen of experiments I have tried. Now obviously, this is just looking at some of the worst artifacts (propeller effect). Also, I obviously can’t really show the real improvements through videos like this since it’s VR and the smoothness just won’t show in videos. But I hope we can confirm the results are better across the board before we can release it. It’s difficult when some of you are very picky :wink:

36 Likes