Nvidia Image Scaling NIS and VR

@Kingofclubs4680 and @Pilotpete123

Let me clarify the story of NIS and driver support.

First let’s clarify GPU support

NIS is simply a graphics shader - this means a program that runs on the GPU and performs operations on every pixels of an input image, to produce an output image. Think of it as graphic shader = GPU program.

Graphics shaders have become a pretty generic thing now, almost like traditional applications: any processor (here GPU) with the right capabilities can execute them.

So remember how there used to be things like “some CPU don’t supports SSE, and hence cannot run SSE optimized applications”? Well that is also true for GPUs now, except the “thing” that indicates compatibility is called the “Shader Model”.

So as long as your GPU supports a Shader Model that is sufficient to run the NIS shader (program), then your GPU is capable of doing NIS.

From the NIS doc itself:

So here it is Shader Model 5.0 is the requirement. And this version has been standard since pretty much DirectX 11.

So for short, as long as your GPU – OF ANY BRAND – is compatible DX11, then it can perform NIS.

Now let’s talk about Driver support

The GPU driver is NOT really involved in NIS. I mean yes the GPU driver eventually allows the execution of our NIS graphics shader, just like it enables any other kind of shaders implemented by the game itself (eg: TAA, motion blur, and other visual techniques).

So what is this NIS option in the GPU driver, and why is it only on recent NVIDIA drivers?

There are 2 traditional ways of performing NIS:

  • One way requires the application to invoke the NIS graphics shader itself, meaning the application is NIS-enabled. This does not involve the NIS feature of the GPU driver at all.
  • Another way is to enable all applications including the ones that are not NIS-enabled to still run the NIS graphics shader just before the display. This is what the NIS feature in the GPU driver does.

The problems with the GPU driver approach are:

  • It’s only supported for NVIDIA drivers, since it’s their technology. But don’t get it wrong, in theory ANY GPU driver could implement the addition of the NIS graphics shader.
  • It only works for contents displayed on computer monitors - because this is where the driver inserts the execution of NIS. It does not work for VR.

The software that I developed (called “application layer”) is a hybrid of both approaches: it is implemented on the “application side”, technically sitting between the application (eg: FS2020) and the OS, and it functions just like the GPU driver option, meaning that with applications that do not support NIS themselves, the NIS OpenXR software will invoke the NIS graphics shader just before handing off the images to OpenXR, which then sends those images to the headset display.

Anyway, a picture is worth 1,000 words, showing the various ways of making NIS:

I hope it clarifies things.

PS: The reason I put “quality is inferior” for the GPU driver and application layer approach is because the games does not know that NIS is happening, meaning that some things done by the game might violate the best practices of NIS, hence resulting in lower visual quality than when the games does NIS itself in the best possible conditions.

14 Likes