Im planning on starting a new project. As of now, I fly a mix of 2D desktop and VR. I love VR but I’m missing the physical controls and haptical feedback of buttons and switches. So it came to my mind to try and build a hybrid A320 cockpit. Kind of like in this video (Mixed Reality + fully physical fighter jet cockpit simulator: The best implementation of XR/MR - YouTube) but this approach is probably beyond my budget.
What do I mean by a VR hybrid cockpit: For me it is a cockpit, that has all the nececary buttons for an A320, but no screens since the screens would be visible in VR. It is important that the cockpit with its switches has the right (original) size and dimensions, since it has to match the virtual world in the VR headset.
For now my plan is to 3d print most of the parts. There are various models available on thingiverse for free or on other sites like cults3d for a (sometimes smal) fee. These combined with electronics from different shops and Arduino to read and manage the inputs, which are then passed on to mobiflight and interact with MSFS. Also JeeHell came up, but I havent looked into it yet. Further on I will focus on the Fenix A320 because in VR the cockpit seems to have the real dimensions.
For VR i will use my Reverb G2 combined with a Leap Motion Controller to visualize my hands in VR, even though the input will come from the physical cockpit and not the “virtual touch” from Leap Motion. If I use it a lot, i will think about an upgrade to the VR headset (Pimax Crystal looks interesting) and an according graphics card (I have a 3080 at the moment).
For now thats all in my head as i have not started building anything yet. I have ordered a 3D Printer (Ender S1) and a few electronic components.
Now I was wondering, if anyone around here has planned or built something similar and what findings and lessons were learned?
For me, especially the following questions came up:
How to adress the positioning issue, so that the virtual and physical world line up correctly? Especially since for example the cockpit of the FBW A320 seems too large in VR, while the Fenix looks a lot more realistic
How do I get the EFB to work in VR without the mouse?
Whats the best architecture to integrate all the switches to the Arduino(s): Does it make sense to go with a matrix approach?
I’m happy for any hints, ideas or critical questions from you!
From the article
"*Operating a mixed reality system necessitates an enormous degree of computer resources. Accordingly, the Redbird system utilises dual GPUs, to help the user get a great view with each eye. The real-world visuals the user comes across are created by using the image creation software offering of Mantis by Quantum3D. They are not controlled by the simulator programme from Redbird.
According to Quantum3D CEO Murat Kose, Mantis is currently being used for military activities. Notably, Virgin Galactic has started relying on a Quantum3D system for suborbital flight pilot training. Quantum3D MR is utilised by a German-based air ambulance business for Airbus H145 pilot training. In addition, organisations dealing with firefighting operations are also relying on technology.*"
Very interesting, although they also go for the mixed reality approach. Of course MR would be a lot more interesting, but also a lot more expensive and at the moment more aimed at professional use cases.
My approach go more in to the direcation of virtual reality, but with the help of some physical assets which cannot be seen from the VR but because of that need to be alligned quite perfectly.
Thank you, yes that is indeed useful, especially since the guy in the video later on built a mockup OVHD panel for an A320: FS2020 Linda FBW320 - YouTube
The ideal would be the switches and panels without any wire or hardware to the PC, only the VR Headset cameras reading and watching your hands inputs in that real physical controls and switches.
I agree, but at the moment I think a feasable solution is, to take the input from the physical switches as the monitoring of the hands is not quite precise enough to notice little buton pushes… but the more important it will be to allign the physical buttons to the virtual buttons, especially e.g. the MCDU.
I tried to operate the EFB with the virtual hands, 3d printed a holder for my LeapMotion to the Reverb G2. It worked but not as easy and quick as one would hope.
