You have arrived at the same conclusion as me, I am also flying with left hand on joystick (like an airbus sidestick) and right hand using VR controller to interact with everything else. I find this much more immersive than using a mouse or the same HOTAS keybind on every plane!
The main problem I have with this setup is throttles on multi engine aircraft. Whilst the throttles on the original Asobo multi engine aircraft (eg 747) move together when grabbed with VR controller, this does not happen on most newer first and third party aircraft such as the A310, ATR 72, etc. I made a bug thread on this, please vote/contribute for them to fix it ! Cannot move both throttles together with VR controllers on many aircraft
As a workaround I am also using a physical throttle on these aircraft, but it is not ideal as then I have to put the controller down and fumble for the throttle, which breaks the immersion and the benefit of the VR controller in the first place ![]()