Nvidia Image Scaling NIS and VR

No, running it through the Oculus and AirLink.
Will test in SteamVR, and I’m guessing I would need to run it through the VirtualDesktop to get it running through SteamVR’s OpenXR runtime.

Update, so MSFS doesn’t have the “openvr_api.dll” to replace with the modded one, so this won’t kick in in the headset itself, regardless whether its Oculus or SteamVR I would run it through.

NIS is enabled only in the desktop mode via Nvidia settings and not the mod, so it affects only pancake images.

1 Like

I’m glad you double checked for us but I think I unfortunatly, was right on this answer which say msfs2020 is openXr not openVr

I hoped i was wrong through…

Interesting… I thought it mirrored the screen.

Nvidia website states Image Scaling is not supported in SLI Mode, Surround Multi-Monitor Clone/Duplicate Mode, Tiled Display, Direct Display, VR, and Integer Scaling mode.

I also try NIS. Because MSFS is a borderless window (not a seperate game screen), I have to set the desktop resolution to Nis upscaling first and then set MSFS display to the same (Lower) desktop resoltion. Upscaling from 1920x1080 to 2440x1440 on my 2k monitor gives the same FPS as native 2440 resolution… :thinking:
I am on the public beta MSFS. Although NIS works (overlay is show at green), somehow MSFS is not programmed for NIS upscaling (maybe because of the renderscale slider in settings does the same?)

Edit: I got the tip i was on CPU limited. So i set the CPU sliders settings lower and the GPU settings higher. I got a 10% better framerate performance (upscaling from 2176 to 2560). So it works but only if your on GPU limited. check it with the in dev mode onscreen FPS counter!

System 9700K 32GB 2080RTX nvidia driver 497.09

1 Like

Hey all,

It’s your favorite OpenXR developer MattB here, bearing gifts!

I’ve spent a few hours trying to get NIS to work with VR, and I have an alpha-quality piece of software for you to experiment with! I’m hoping the community will find this useful and improve your gaming experience!

(updated link to reflect the latest version)

Q: What is this?

It’s a piece of software sitting between an application (here MSFS) and the operating system (the OpenXR runtime) and intercepting what’s rendered by the application, running the NVIDIA Image Scaling algorithm, then submitting the upscaled output to the operating system. Effectively it’s another render scale, but the NIS algorithm also performs sharpening in an attempt to make the visuals more crisp despite the lower resolution.

Q: Does it work?

I think it does, but I’m counting on this forums’ community to test it out and tell me what it’s really worth. I’m able to see an increased FPS when turning it on (+ 10-15 FPS on my machine), but I am not too sure if the visual quality is on par (y’all have a much better eye than me I’m sure). What I can say for sure is that between a cheap bilinear upscaling and NIS, the quality is definitely superior with NIS.

Q: Should I try it?

It’s a very early prototype that I’ve spent only a few hours working on. So it could be rough.

If you’re not too experienced, you probably shouldn’t try it until we figure out that it’s worth it and I work out the quirks.

If you expect to install it and get a jump in FPS for the same quality out-of-the box, this is also not for you and you should check back later after it’s tweaked.

Q: What if it doesn’t work?

If it doesn’t work for you specifically, we can try to debug as much as possible, but every PC setup is different so I can’t guarantee I will be able to resolve all problems…

If it doesn’t work at all (performance isn’t as good, quality is too low), then I’ll consider just killing the project and call it a failed experiment.

Remember, I am delivering this as an experiment, no guarantees!

Q: What are the best settings for it?

I don’t know and since this community has a history of being really good at figuring out combinations of settings, I’d like to crowd-source the answer to this question :slight_smile:

Q: Does it work with DX11 and DX12?

Only DX11 at this time.

Q: How does the scaling value fit in with the OpenXR render scale and the game’s render scale?

The software uses the resolution returned by OpenXR (ie the one after OpenXR scaling, that you can see in the OpenXR Developer Tools if using Windows Mixed Reality, under System Status → View configuration). It applies its scaling to that resolution. The actual resolution can be observed in the log file (see README).

You’ll see something like:

Scaled resolution is: 2206x2161 (70% of 3152x3088)

The NIS scaler should replace any in-game render scale setting that was being used before. So if you used 80% as the in-game render scale, I suggest you set that back to 100%, and use 80% with the NIS scaler instead.

Q: How do I experiment with the various settings?

There are only 2 settings mapping to the NIS settings: scaling (between 0 and 1) sharpness (between 0 and 1).

Changing the scaling requires you to exit VR, modify the config file, then re-enter VR. The config file is named FS2020.cfg and it must be in the same folder you copied the software (DLL). By default I’ve put 70%, but no idea if it’s the best value.

Changing the sharpness can be done in increments of 5% by pressing Ctrl + Down arrow (or Ctrl + F2) to decrease and Ctrl + Up arrow (or Ctrl + F3) to increase. By default, I put 50% but it may not be the best value. The new sharpness value can be observed in the log file (see README). When satisfied with your tweaking, you may then modify the config file to make it permanent.

You can use Ctrl + Left arrow (or Ctrl + F1) to enable/disable NIS and switch to a bilinear scaler (cheap scaler) instead, so you can see the improvements that NIS provides.

Q: What headset does it work with?

It should work with any VR headset thanks to OpenXR. We’ve seen success with Windows Mixed Reality (eg: HP Reverb), any headset going through the SteamVR runtime (Valve Index), Pimax, Oculus Quest…

Q: What GPUs does it work with?

I think the NIS shader works on any modern GPU, even not NVIDIA. I have only tried on a GT 1030 (not with MSFS obviously) and a GTX 2080.

Q: Will it work with other games like the ones from Steam?

This only works with OpenXR applications, not OpenVR and the other ones.

Even with OpenXR applications, I can’t guarantee it will work as I’ve only implemented the bare minimum for MSFS.

Q: Why NIS and not DLSS?

I don’t have an RTX card.

Q: Why NIS and not FSR?

I have looked at both SDKs, and the NIS shader was super easy to integrate with Direct3D 11, literally copy a few files then make 4 functions calls. The FSR code was more complex so I didn’t want to invest time in it.

Q: Why are you developing this?

Mostly for fun and learning, and to see if I can improve this community’s experience.

Q: Is it open source? Can I contribute?

It’s 100% open source, and if you’d like to contribute I’m happy if you get in touch with me!

Q: Is this affiliated with Microsoft?

While I am a Microsoft employee working on OpenXR, please note that this is a personal project not affiliated with Microsoft.

Q: Why does this have a weird name?

It’s following the OpenXR naming convention:

XR_ → This is for Khronos’ OpenXR (as opposed to GL for OpenGL or VK for Vulkan for example).
APILAYER_ → This is an API layer, ie it intercept existing API, as opposed to an EXT that would add new ones.
NOVENDOR_ → This is not owned by any company, I am developing this indenpendently.
nis_scaler → That’s what it does, duh!

Q: How does it work?

NIS is a post-processing algorithm that NVIDIA integrated to its drivers so that in theory any application rendering to a monitor can benefit from the upscaling. But that’s not how VR works, because VR is a very special way of handling a monitor, so the driver just doesn’t seem to handle it.

Instead, what this software does is intercept the OpenXR calls from the application, perform the NIS processing pass, then forward the output to the real OpenXR runtime.

Step 1 is to make the application believe that it must use a lower resolution. When the application calls OpenXR to query the resolution, we just return the downscaled resolution (lower resolution).

Step 2 is to intercept the textures that OpenXR passes to the application for rendering, and instead to create smaller textures for the application to use. We keep a hold of the real texture that OpenXR needs (the one at full resolution).

Step 3 is to intercept when the application submits its rendered texture. When this happens, we invoke the NIS shader to upscale the texture (the smaller one) and write its output to the real OpenXR texture (the one at full resolution). Unlike what your driver typically does with NIS, it’s useful to notice that we must do the upscaling twice: one upscaling for each eye! We can then tell the OpenXR runtime to take it from there (ie use the upscaled texture).

29 Likes

Wow, thanks for your trouble. I’d love to try it out soon! I’m running a Valve Index right now, so I’m not sure if OpenXR is even installed / applies to me. If I manage to get it running, I’ll surely report back with my findings. We desperately need to be able to introduce some scaling (NIS/DLSS) or at the very least some sharpening filter for FS2020, and that’s coming from someone running a 3080…

You are for sure using OpenXR since this is the only VR API that the game supports! Think of it as OpenXR being the DirectX of the VR stuff. But of course the game also still use DirectX for other stuff on top of VR!

Now I’m definitely curious to hear how it will do with an Index and if the game will start at all! If not, let’s talk and we can try to debug together.

Yeah, i was reading up a bit on it and I think I understand where the layer sits. SteamVR shows “Current OpenXR Runtime” to be “SteamVR”, so I suspect SteamVR has its own bundled version of OpenXR.

Right now I’m not seeing any OpenXR log appear (to check if the application name is ‘FS2020’ indeed. So that would mean either a) nothing is being loaded/configured and your script is simply targeting a runtime not being used in that location/key, or b) the log file is simply in a directory which I haven’t been able to find yet.

In the hopes of b), I simply put your ‘FS2020.conf’ file in the directory with some values that should be visible to me, but unfortunately nothing happens yet.

I’m going to be searching for the right OpenXR directories and logs for the SteamVR variant (at least, operating from the assumption that that is actually a thing…). I’ll report back if I find anything. :slight_smile:

In theory the software should get loaded regardless of the SteamVR settings and the OpenXR runtime, as there is a common component called the OpenXR loader that should take care of that.

The log directory is in fact dictated by the application (MSFS). You should be able to open Start menu then type in directly %LocalAppData% to get there.

BUT I am also now realizing that I used the MSFS version from the Microsoft Store and not Steam for testing, so maybe the log file end up somewhere else because of Steam. I guess you could do a system-wide search since the file name is always XR_APILAYER_NOVENDOR_nis_scaler.log.

You technically don’t need this log file, but it will help confirm that the software is running or debug other issues. Have you tried Ctrl + F1 to disable NIS scaling? It shows a pretty visible difference for me, especially for far objects, and would confirm that you installed everything properly.

Okay, I’m a bit further along:

  • SteamVR ‘OpenXR’ ActiveRuntime is C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json for my setup. But in the end this isn’t really relevant.
  • Problem: The registry keys were not being added because windows (PowerShell) wouldn’t execute your script because of an execution policy. Solution: I’ve manually added the keys instead.
  • Now the log file appears in the directory %LocalAppData% showing all the good lines like XR_APILAYER_NOVENDOR_nis_scaler layer is active.

However: The FS2020 VR mode never activates anymore. I can keep pressing Ctrl+Tab, but it immediately stops and returns back to SteamVR home. I’m going to see if I can find a log file to tell me what is wrong. Perhaps it’s something relating to an unsigned piece of code again. Could that be the case for the DLL which I assume should be loaded now?

Edit: Some added context:

  • Using windows 11
  • Using Microsoft Store version of FS2020

Good progress. If you’re seeing the “… is active” message then the DLL is loaded. I’m thinking there’s something failing during the initialization of my code and that is specific to the Index runtime.

When you do Ctrl-Tab is when I intercept the graphics initialization and do extra steps to setup the NIS scaler. This involves substituting Direct3D resources which can be tricky and need small variations between each OpenXR runtime (since I’m doing stuff that isn’t very “standard”).

I think the only way to know is for me to give you a version with extra logging that will tell us specifically where it fails. I’ll do that tomorrow (Saturday). Thanks for trying, and I think this is pretty much expected glitch :slight_smile:

1 Like

I just installed your tool.

With RTX3080 on a Reverb G2 i was able to maintain stable 27-30 fps with 60% internal rendering and 200% in WMR.

Now with the same setting i’m getting 45!
There is slight loss in quality, but the smoothness is worth it.

Thank you!

1 Like

Promising project.

The thing I don’t understand though, is how is this different from just modifying the OXR render scaling? I

I admittedly do not fully understand the new NIS feature since I never fly in 2D so I have never tested it out. But doesn’t it just feed a downscaled image to the sim which then upscales it?

Aren’t we already doing the same thing with the existing OXR render scaling?

How is using your tool to further downscale the feed before the sim sees it any different than just lowering the OXR render scaling?

Just trying to understand it better!

2 Likes

ok, i don’t think the Install-XR_APILAYER_NOVENDOR_nis_scaler.ps1 powershell script is creating the key it needs to so i have tried to create it manually, is this what it should look like?

Almost, except the key ‘$jsonPath’ should be the full path to json. In my case:

C:\Program Files\OpenXR-API-Layers\XR_APILAYER_NOVENDOR_nis_scaler.json
1 Like

I think, it’s more related to SteamVR, cause I’m running with an HTC VIVE PRO 2 and I got the same problem.

Windows 11
HTC VIVE PRO 2
Windows Store MSFS2020
RTX 3070

Here the log :

dllHome is “C:\Program Files\OpenXR-API-Layers”
XR_APILAYER_NOVENDOR_nis_scaler layer is active
Could not load config for “FS2020”
Could not load config for “Zouna”
XR_APILAYER_NOVENDOR_nis_scaler layer is active
Loading config for “FS2020”
Advertise all texture formats
Prioritize capable formats
Use scaling factor: 0.700
Sharpness set to 0.500
Scaled resolution is: 1769x1769 (70% of 2528x2528)
Using indirect texture format

My log with Pimax 8kx:

dllHome is “C:\Program Files\OpenXR-API-Layers”
XR_APILAYER_NOVENDOR_nis_scaler layer is active
Could not load config for “FS2020”
Could not load config for “Zouna”
XR_APILAYER_NOVENDOR_nis_scaler layer is active
Loading config for “FS2020”
Advertise all texture formats
Prioritize capable formats
Use scaling factor: 0.500
Sharpness set to 0.800
Scaled resolution is: 1766x1598 (50% of 3532x3196)
Using indirect texture format

But pressing ctrl+tab to get into VR does nothing.

Thanks all for the feedback!

So it looks like we only have success with WMR, which isn’t surprising given that it is all I’ve tested with!!

I’ll test with SteamVR today and see if I can identify the issue the other users have reported! Otherwise I’ll publish a debug version with extra logging so I can get better error reports. Thanks for your patience!

Also it looks like the powershell install script is a hit or miss depending on people’s policies. Will see how I can resolve that.

The first part of the process is the SAME as the OpenXR render scale. However after that part, instead of letting the OpenXR runtime upscale the image using a simple algorithm, it uses the NIS algorithm which provides sharpening as well, which tries to give the same quality as the full resolution. If you can get the software to work, there’s a toggle (Ctrl + F1) to switch between regular upscaling and NIS, which shows the difference (blurry image with regular upscaling and a sharper image with NIS).

1 Like

@Borostwory1986 - glad to hear it’s working for you!

May I ask what settings you ended up with for scaling and sharpness?