It used to be that way. I would get out of VR and it would crawl for a while before releasing FR back - unless I closed the MXR Portal app (which will release it immediately).
But now after exiting VR FR is immediately good, but I can never get back to VR.