OpenXR NIS upscaling software - Release thread

Maximum kudos to @mbucchia. Just tried the NIS scaler and it works brilliantly for gaining FPS. I had FS2020 tuned for decent image quality at 100% native resolution of my HP Reverb G2. But I had to trade off FPS for visual quality meaning I was only getting around 25 FPS (RTX 2080ti overclocked). But with the NIS scaler set to 80% render scale and 50% sharpenss (100% render scale in FS2020 and OpenXR custom render scale enabled and set to 100%) I am now getting 34 FPS without any observable loss of visual quality.

Thanks for making this. Now I have the VR flying experience I was always hoping for with FS2020.

1 Like

Well at this point you are putting A LOT MORE effort into MSFSā€™s VR support than the Asobo does. I donā€™t remember if you posted a ā€œbuy me a coffeeā€ link anywhere, I wold suggest putting it in the actual UI of your next release for this plug-in.

Here is a shot in the dark, since you are looking to expand the OXR functionality of your plugin.

Currently in Oculus software there is an option called
ā€œFOV Tangent Multiplierā€. If Iā€™m understanding correctly how it operates, it simply sets the percentage of the HMDā€™s screen to be rendered. This directly translates into a smaller resolution image to render, with the caveat that you are effectively decreasing your FOV.

I donā€™t know if something similar is available directly through OXRā€™s API or if its an Oculus exclusive thing, however it is VERY effective in bumping up FPS. At 70% (Iā€™m assuming its 70% as Oculus software sets 0 for NO crop and I run it at 0.7 both vertical and horizontal), I am gaining a noticeable boost in FPS (havenā€™t recorded the exact numbers, but its clearly visible in the performance graph when I play with it).

The problem is that since it cuts down the rendered image, we are left with a black border and depending on how far you go with these settings and how thick the actual padding is of the HMD, you start seeing this black border.

Would it be possible to do something like what I tried to illustrate bellow, where we can still control the percentage of the screen that gets rendered, but instead of getting a black border, extend the pixels of the rendered image, outwards to fill in the black area? Maybe would need some kind of a heavy blur filter on top as it would probably be very flickery during movement.

3 Likes

For those interested in understanding the difference between NIS done by the application, NIS done by the GPU driver, and NIS done by the application layer, hereā€™s a post I just made on the ā€œoldā€ thread:

1 Like

Thanks for bringing this up. The first part of your suggestion is already on my ToDo (custom FOV, black borders). Weā€™ll experiment with that, and based on the effectiveness of this reduced FOV, we can then consider the 2nd part. We did a quick experiment with the reduced FOV in some other thread on the forum, and it did not seem to help very much. So my hopes are not too high, but with the new toolkit framework we should be able to experiment more easily!

2 Likes

I do not use discord but Iā€™ll look. BTW by website I really just mean a single GitHub page that is more welcoming that the current one :D, one of those GitHub.io page where you will have links to the download, the tutorial videos, the FAQ (and discord). So I donā€™t think that ā€œwebsiteā€ will take long to make.

Thinking about this but there are a few issues with it I think. It would certainly make my code MUCH simpler :smiley: but I am also worried it might break for folks not using the Leap software at all. I need to experiment.

Iā€™ve grabbed a quick test.
As you can see, using Oculusā€™s settings, there is a major impact on the FPS from the FOV settings.

70 FOV is what I use, I can barely see the borders start to peak over the foam frame of the HMD itself, but I can live with that, given the boost in FPS.
Anything bellow is unplayable, as it starts to looks like a hole in the wall.

Potentially by removing that black border, FOV reduction could be pushed further, since it would still be mostly in the peripheral view and if there is still light and movement in that area corresponding to the central image, might be enough to make it somewhat less distracting.

3 Likes

I can also confirm it improves visuals, also increasing 2FPS for me in VR. :slight_smile: Additionally, I tested the impact Iā€™m having for each graphic setting in FPS and this is what Iā€™ve found.

image

Funny how much clouds impact performance.

By Setting my own optimized settings I manage to run VR at 36+fps over San Francisco with good visuals (ā€¦x2782) with Reverb G2, 100OXR, 90%NIS and 100%TAA. rtx3080ti and 12700k.

Update: Turning VSynch OFF in NCP fixed my micro stutters issue. I just found my sweet spot :blush:. For micro stutters, this helped me (along with disabling hypertheading after the game launches in process lasso and prioritizing background apps in windows settings): https://www.reddit.com/r/flightsim/comments/ifle4x/huge_performance_booststutter_reduction_by/

It feels the game is still not fully CPU optimized since I was getting unstable micro stutters from session to session.

8 Likes

Has anyone tried this with DCS and gotten it to work? Looking for help if you have.

DCS does not use OpenXR AFAIK, so it wonā€™t work. Though you should be able to use openvr_fsr with it.

1 Like

Great work - I got about ~5 fps boost and I can finally see the taxi signs. The only small downside is that I notice the shadows (in cockpit) move a lot, but itā€™s worth the sacrifice. Thanks!

Yep DCS via OpenVR and this with the NIS options enabled works great:

OpenVR_fsr has a nice ā€˜radiusā€™ setting that might work well for the future OpenXR revisions?

1 Like

The radius (technical term would be ā€œfixed foveated renderingā€) is one of the 20 items in my To-do to investigate later. It might bring a few FPS back but I donā€™t expect too much.

4 Likes

I think it works well in flight sims especially, because we want/need extra clarity where we look at tiny instruments, but can get away with cheap bilinear in the periphery.

When I first released alpha-4 I reached out to a few and asked them to run a special option that captured statistics. Ths results were that the NIS shader was taking something like 700-800 microseconds to run for the full stereo frame. Doing the fixed foveating will reduce that number (less pixels to process), letā€™s assume it will cut it in half. So ok letā€™s put us at 400 microseconds. Hereā€™s the thing, saving 400 microseconds per frame at 40 FPS, thatsā€¦ gaining under 1 FPS. Even if somehow the NIS time is cut to 200 microseconds(!) at 40 FPS this means still just 1 FPS.

So on paper, this does not look promising to me.

To explain the math: at 40 FPS it takes 25ms to render a frame. Remove 400 microseconds from that, thats 24.6ms. Then 1/24.6 = 40.65 FPS.

6 Likes

Good info, understood. As a layer you canā€™t really get the optimizations of viewports that would allow different resolutions where a fixed foveated would shine. Itā€™ll be interesting on how well the Varjo Aero works with MSFS, with its foveated approach.

It wonā€™t make a difference. You need game engine support to do foveated rendering.

To work with the Varjo Iā€™m pretty sure the game needs to implement support for this: XR_VARJO_quad_views+XR_VARJO_foveated_rendering: The OpenXRā„¢ Specification

1 Like

Early reviews describe MSFS being especially clear with it, so not sure how thatā€™s working with the hardware foveated 4 viewports it uses. Perhaps something within their implementation but not in MSFS?

The NIS timing will be reduced by 400us but wouldnā€™t the real gains happen on the game side, it will be making less pixels too? for example normally it renders 2000^2 and now it has to render 1500^2, thats nearly half the pixels where the game does not have to generate terrain, clouds, buildings etc.

edit- thought of a downside to thisā€¦ if you ā€˜foolā€™ the game and tell it the canvas is 1500^ and it draws that assuming its the full frame, and you display it at 2000^ with filled-in edges, things are going to look smaller

No it wonā€™t. The game still renders all the pixels. What you are thinking of is reduced FOV (black edges).

1 Like

Iā€™m not sure what issue you are talking about, but lighting is not something that can be tweaked outside the rendering pipeline of the game. So I donā€™t think thereā€™s anything I can do here.