Looking for advice, changing from VR to "not VR" is tanking my framerates

This is written here:

[Read First] VR Known Issues & Workarounds - Virtual Reality (VR) - Microsoft Flight Simulator Forums

Known Issues (content):

  • The sim’s performance may be poor with the Mixed Reality Portal running in the background when not in VR mode.
    • Workaround: Close the Mixed Reality Portal when not flying in VR mode.

As for switching back and forth, beware, sometimes it fails re-entering back into VR. It is happening sometimes to me but in general I’m also changing OXR settings when I do and this could be the main reason.