SOLVED!
It was turning off Hardware Accelerated GPU Scheduling, thanks to a reddit user hint!
edit: i now get about 20ms/more than 40fps in NYC with DLSS, many settings on High or Ultra, render res 3700x3700 on vive pro 2, with my 4090/5800x3d/32GB CL16/… , very stable, except for some microstutters when looking to the sides.