I have an i9-10900k, 3080, 32 GB of RAM, gigabit connection. I get 60 fps when everything is turned to low. As soon as I turn up object/terrain level of detail from 50 my fps turns to utter ■■■■.
Unplayable! I can either play the game in VR with it looking like a grainy, blurry mess or not play at all. While in 2020 I could go up to ultra clarity with OpenXR etc, just like you.
Idk what they did to get performance to be this ■■■■■■, but you are not alone. VR performance is awful for most.