I have a beastly rig myself, and I fly VR all the time on Medium or lower. I get a solid 40fps by being GPU bound.
See this classic settings post on how to load up the GPU while not loading the CPU. Set the frame rate to 30 or 40, and load up that GPU.