I think a built-in scratchpad/kneeboard is a missing, yet actually must-have functionality for almost all current major platforms. Unlike when flying with a flat-screen, taking notes (clearances) is very difficult when using a VR headset. Taking the HMD off and on is not only annoying but also could sometimes break the mounting position of the HMD on your head that you’ve adjusted for getting the best sweet spot/FOV, or make some fogs on the lens, as well as killing the immersion. So I think it would be nice if MSFS2020 native VR could have this function included, as an essential part of a more realistic and serious experience.
To be more specific, I think we need a VR kneeboard that could do some of the following:
- Let us take notes (“the scratchpad”), ideally by handwriting(using a pen tablet, tablet, or even your phone; and of course, the VR controller or mouse), or by keyboard for those who don’t have extra external devices.
- Display PDF or other types of local files for displaying the charts and so on.
- Display browsers, playing videos and music, and so on.
- Basically overlay windows of the desktop and/or the “background” desktop windows can achieve all the above and more but it has to be:
a.The overlay window must be relocatable and resizable.
b.You can interact with the overlay window without changing the focus (for example, don’t need to hit alt+tab)
I know there are some third party VR overlay software can achieve the goals, but those may have compatibility issues as I’ve heard some of them had for DCS.
I’ve also heard some built-in functionality of the HMD can do the overlaying (Oculus and WMR) but it seems they may have some glitches or insufficiency. But I can’t tell since I currently can’t test it myself.
So my suggestion is either to make it in-game functionality or make it compatible with those third-party solutions.
(Also plz forgive my poor English.)
Updated thoughts 2021/07/25:
Although currently some awesome products on the market that can achieve basic kneeboard+scratchpad operations, giving that Asobo has stated they will make some major improvements for the VR part in the future, I’d still like to see a VR kneeboard that allows us to accomplish basic kneeboard tasks being provided natively.
Ultimately, a desktop(background) window overlay system should be a “one size fits all” solution due to its versatility and infinite possibility, though it might not be something easier to be achieved. There are products like OVR Toolkit available for such functionality, but I think a native solution made with the collaboration with Microsft’s own WMR team might just better address the changing of focus issue when switching the focus between the overlaid window and the cockpit(“the sim world”) or at least make the interacting with the overlaid window more seamless.
All in all, the idea is, having something that does what a real kneeboard can do. (Though giving the digitalization nowadays, we may eventually need the functionalities that an EFB/EFB software(such as foreflight) can provide.) Well, what they can do/provide?:
Displaying materials: IFR charts, VFR sectional charts, any paper products. To achieve this, we need a file(pdf, jpg, txt…) display system that can be easily managed and navigated through in game.
Writing down: take notes on a blank page, of course, most kneeboard addons out there can do this; but also, have the ability to write down/draw ON the aforementioned materials! Sometimes pilots IRL would make some notes/marks on the charts during planning, preflight or in-flight; or they just need to write down something very quickly and that chart is just the one readily available; or they have to modify the marks on the VFR chart that previously made before the flight; or they need to fill the navigation log when they doing the navigation in an old-school fashion or training.
A possible solution could be: make the background image of the scratchpad page customizable by a jpg image, pdf file, excel sheet…or combine 1 and 2 all together: resizable windows in VR can display a file in the disk, and let the user drawing and erasing on it. Ideally, it should be able to display at least two of these windows at the same time(one usually has two knees:) ), and each window contains several pages that can be flipped.
Other than the functionality part, another important aspect is the interaction part. As mentioned before, it is good to have the transaction of the cursor between the VR sim world and the kneeboard window seamlessly without changing the focus: you can just move the cursor into the window to interact with its content or drawing and move out, and still be able to fly at the sometime. Moreover, drawing with a mouse could be painful, so it will be nice to have the tablet/pen·graphic·drawing tablet support, just like the VRK addon in the DCS world. VRK - a virtual reality enabled kneeboard with touch and ink support - Mods and Apps - ED Forums
The next good stuff that would be much appreciated is… a calculator! Be it a “Whiz Wheel” (flight computer like the mighty E6B, a digital flight computer, or just a regular number pad calculator, or both, it will be much needed for old-school flyers or a student pilot that like to take the advantage of the sim to practice more.
I hope these thoughts could be somewhat useful for Asobo to make the VR and the sim better and better, more and more real. Thank you very much!
Some videos from Foreflight, explaining its multiple template scratchpad, and annotations function for maps, charts, documents. I think they could be good references for making a good “digital scratchpad”.
But I still believe it is an easier way to do it, if the background of the scratchpad is changeable or drawing can be done on every “window(chart viewer, pdf viewer, etc)”. That way you can make all kinds of modifications by yourself to achieve similar functionality.