SU14 and Reprojection Mode: What's really going on?

[mbucchia]

@AcidBath6450

Considering that “always generating frames when you don’t hit frame rate” implies that the VR system attempts to enforce a constant 11.111 ms screen refresh duty cycle for a 90 htz HMD

This is how displays work. They have a scheduled scan-out at regular interval programmed in the hardware, there is no way around it.

@AcidBath6450

“best effort”
0 ms - first frame from app displayed
11.111 - generated frame displayed
20 ms - app frame displayed
31.111 ms (or 33.333?) - generated frame displayed

This isn’t how it works no. You can only display at 11.111, 22.222, 33.333. So your frame at 20.0 is “latched” for later display. Then at 22.222 the frame is spatially reprojected with the latest headset pose to reduce the juddering.

@AcidBath6450

“enforce 90 fps”
0 ms - first frame from app
11.111 - generated frame displayed
20 ms - frame received from app
22.222 - app frame displayed (2.222 ms latency)
33.333 - generated frame displayed
40 ms - frame received from app
44.444 - app frame displayed (4.444 ms latency)
55.555 - generated frame displayed

Yes, this is more like it. Except that the latency typically won’t accumulate as you’re saying. This is because the VR stack will possibly make you “sleep”/wait in order to reduce the latency and avoid accumulation.

Unfortunately the strategies to avoid this accumulation may be more or less well implemented,., you are on WMR so you might be familiar with the “Turbo mode” or the “Prefer framerate over latency”. What these options do is literally avoid any sleep/wait and request to pump frame as quickly as possible.

@AcidBath6450

If “enforce 90 fps” is happening the app’s frame display latency grows and grows until the resync. I think that this certainly results in judder.

With what I told you above, you can now go and compare with/without Turbo mode. Tell me which one you prefer. Turbo on WMR has historically helped increasing framerates dramatically, sometimes 2 digits, at the cost of this added/accumulated latency. IMO it works a lot better when not worrying about that accumulation.

@AcidBath6450

Given if any of these cases are happening, a remedy for this VR judder problem is easy: either run at 90 fps (dream on), or a whole fraction there of, e.g. 45 or 30

You got it. This is why locking at 45 FPS feels better than running at 50-55 FPS. You are making the frame pace and the amount of spatial reprojection happening consistent for every frames.

EDIT: Here is another diagram I made a while ago about spatial reprojection (LSR):

• The application is not able to keep up with the headset’s panel frame rate
• For each missed frame latching, the late-stage reprojection (LSR) reuses the last submitted frame
• The LSR queries the most recent tracking information
• The LSR performs a simple depth reprojection (when the application submits depth information) or auto-planar reprojection (when the application does not provide depth information)
  • Almost no PCVR application provides depth information to the runtime
• The reprojected frame is used for scan-out
• When the application is finally able to submit a new frame, it has to wait for the next latching opportunity, therefore adding latency