Scratchpad/kneeboard/EFB/windows overlay

I think a built-in scratchpad/kneeboard is a missing, yet actually must-have functionality for almost all current major platforms. Unlike when flying with a flat-screen, taking notes (clearances) is very difficult when using a VR headset. Taking the HMD off and on is not only annoying but also could sometimes break the mounting position of the HMD on your head that you’ve adjusted for getting the best sweet spot/FOV, or make some fogs on the lens, as well as killing the immersion. So I think it would be nice if MSFS2020 native VR could have this function included, as an essential part of a more realistic and serious experience.

To be more specific, I think we need a VR kneeboard that could do some of the following:

  1. Let us take notes (“the scratchpad”), ideally by handwriting(using a pen tablet, tablet, or even your phone; and of course, the VR controller or mouse), or by keyboard for those who don’t have extra external devices.
  2. Display PDF or other types of local files for displaying the charts and so on.
  3. Display browsers, playing videos and music, and so on.
  4. Basically overlay windows of the desktop and/or the “background” desktop windows can achieve all the above and more but it has to be:
    a.The overlay window must be relocatable and resizable.
    b.You can interact with the overlay window without changing the focus (for example, don’t need to hit alt+tab)

I know there are some third party VR overlay software can achieve the goals, but those may have compatibility issues as I’ve heard some of them had for DCS.

I’ve also heard some built-in functionality of the HMD can do the overlaying (Oculus and WMR) but it seems they may have some glitches or insufficiency. But I can’t tell since I currently can’t test it myself.

So my suggestion is either to make it in-game functionality or make it compatible with those third-party solutions.

Thank you!
(Also plz forgive my poor English.)

——————————————————————————————————————
Updated thoughts 2021/07/25:
Although currently some awesome products on the market that can achieve basic kneeboard+scratchpad operations, giving that Asobo has stated they will make some major improvements for the VR part in the future, I’d still like to see a VR kneeboard that allows us to accomplish basic kneeboard tasks being provided natively.


Ultimately, a desktop(background) window overlay system should be a “one size fits all” solution due to its versatility and infinite possibility, though it might not be something easier to be achieved. There are products like OVR Toolkit available for such functionality, but I think a native solution made with the collaboration with Microsft’s own WMR team might just better address the changing of focus issue when switching the focus between the overlaid window and the cockpit(“the sim world”) or at least make the interacting with the overlaid window more seamless.


All in all, the idea is, having something that does what a real kneeboard can do. (Though giving the digitalization nowadays, we may eventually need the functionalities that an EFB/EFB software(such as foreflight) can provide.) Well, what they can do/provide?:

  1. Displaying materials: IFR charts, VFR sectional charts, any paper products. To achieve this, we need a file(pdf, jpg, txt…) display system that can be easily managed and navigated through in game.

  2. Writing down: take notes on a blank page, of course, most kneeboard addons out there can do this; but also, have the ability to write down/draw ON the aforementioned materials! Sometimes pilots IRL would make some notes/marks on the charts during planning, preflight or in-flight; or they just need to write down something very quickly and that chart is just the one readily available; or they have to modify the marks on the VFR chart that previously made before the flight; or they need to fill the navigation log when they doing the navigation in an old-school fashion or training.


A possible solution could be: make the background image of the scratchpad page customizable by a jpg image, pdf file, excel sheet…or combine 1 and 2 all together: resizable windows in VR can display a file in the disk, and let the user drawing and erasing on it. Ideally, it should be able to display at least two of these windows at the same time(one usually has two knees:) ), and each window contains several pages that can be flipped.


Other than the functionality part, another important aspect is the interaction part. As mentioned before, it is good to have the transaction of the cursor between the VR sim world and the kneeboard window seamlessly without changing the focus: you can just move the cursor into the window to interact with its content or drawing and move out, and still be able to fly at the sometime. Moreover, drawing with a mouse could be painful, so it will be nice to have the tablet/pen·graphic·drawing tablet support, just like the VRK addon in the DCS world. VRK - a virtual reality enabled kneeboard with touch and ink support - Mods and Apps - ED Forums


The next good stuff that would be much appreciated is… a calculator! Be it a “Whiz Wheel” (flight computer like the mighty E6B, a digital flight computer, or just a regular number pad calculator, or both, it will be much needed for old-school flyers or a student pilot that like to take the advantage of the sim to practice more.


I hope these thoughts could be somewhat useful for Asobo to make the VR and the sim better and better, more and more real. Thank you very much!


Updates 2021/07/27:
Some videos from Foreflight, explaining its multiple template scratchpad, and annotations function for maps, charts, documents. I think they could be good references for making a good “digital scratchpad”.

But I still believe it is an easier way to do it, if the background of the scratchpad is changeable or drawing can be done on every “window(chart viewer, pdf viewer, etc)”. That way you can make all kinds of modifications by yourself to achieve similar functionality.

I love the ‘kneeboard’ idea. Not so much with the other two. If you’re flying in VATSIM you have to be able somehow to write down your clearances, frequencies, taxi instructions, etc. If you’re in VR, that becomes almost impossible as you mentioned. You do have to be able to access the menu somehow and the drop down menu is a great idea, even if it does get in the way sometimes.

Throttle quadrants are not that expensive, mine is on my yoke already. Not sure what you mean by ‘motorized throttle’. But if you’re going for the immersion experience, then by all means, get yourself a throttle - you won’t be unhappy at all. It’s almost as much of a difference as going from flat screen to VR. It definitely adds to the immersion experience. In XPlane you can move the throttles using your hand controller, just ‘look’ at the throttle in the sim, point with hand controller, click and drag to desired position. Easy. But I still prefer a mechanical/physical throttle.

1 Like

How do you know those options are not already in their VR implementation ?? Voted for it anyway :joy:

Kneeboard is a must for me, and/or something like Avitab too.

BTW, there is a mod for disabling the hints already … but it would be nice having the oficial option.

Actually, I’m not opposing the idea of the drop-down menu, it’s helpfull. Displaying this “hint” icon whenever you move the mouse regardless the position of the cursor, is what I found annoying.

It would be nice to have the option to turn off the displaying of the “hint” (but not to disable the access to the actual menu icons), or make it only be displayed when your cursor is near the top of the screen instead of anywhere.

For the “motorized throttle”, I’m referring to the one that has a motor to automatically synchronize the pyhsical throttle lever to the throttle setting accordingly when the auto throttle is engaged. It’s usually more then 2.5K~4.5K EURO. By having the throttle position indicator, you can manualy synchronize the physical throttle lever to the virtual one before you disconnect the AT.

Surely it would be nice if they already made the kneeboard.
Yes an EFB is good as well for displaying the charts or playing video/music, browsing. It’s actually a better and more immersive way for those purposes.
But I found it seems that the importance of the scratchpad for VR flying has been forgotten by the developers, at least it is the case for the rather well-made native VR mode of both XP and DCS.

It’s nice to hear that there are mods already available for hiding the “hints” arrow. Can I ask for the name of the mod or the keyword to search for? Thank you.

It’s part of the uimod:

1 Like

For me, the above functionality should be created, ideally by Asobo or, at a push if early enough, by a 3rd party, to be used in all future 3rd party aircraft.

As well as acting as a standard EFB for all aircraft, it would also provide a standard UI for aircraft specific settings. Like Avitab, it could either be mounted on a purpose-made bracket, if one exists, or left in the pilot’s door pocket, to be opened up and placed in a moveable position in view of the pilot.

1 Like

After 4 days of VR flying, the Kneeboard is what I’m missing most.

Appreciate this is an old thread and I’m new to the forum so maybe this has progressed further elsewhere… but bumping it anyway.

I just want a VR kneeboard of some kind where you can make notes and view PDF charts. It strikes me that the “easiest” way would be for it to simply be a 3D window like ATC, that can float around for you to place as you need it, type notes, and view charts.

Is there a mod for this as a stop gap?

5 Likes

Maybe try OVR Toolkit. MSFS 2020? :: OVR Toolkit General Discussions

It’s payware, but it’s very good. It can bring windows on your desktop into VR. But since it’s a steamvr app, you have to launch steamvr at the same time. You can find instructions in the link above. However, I currently don’t have the opportunity to test it, not sure the switching between the windows and VR cockpit is as good as in the native steamvr games.

The good news is, the developer is a simmer as well, and very active in developing.

Still, I hope Asobo can realize the importance of a native VR kneeboard that can make notes and view PDF charts at least, and that is supporting drawing tablet so we can instinctively grab the pen and note things down and don’t need to worry about anything also…but maybe I’m dreaming too much

1 Like

I just need kneeboard in VR as irl :star_struck:

1 Like

Fully agree - I would love some way of viewing the Navigraph charts within VR. The ability to see these on a knee-board, or through the top menu, or even as a window floating off to the side would be great. Flying IFR in VR without them is a challenge.

1 Like

Doing it in Xplane, I can say that it is totally feasable to write clearances in VR when doing Vatsim… with the help of a simple graphic tablet.

There is a plug-in called Move VR which allows to open windows such as MS Paint ok xplane. So it makes it possible to see our handwriting in VR and taking clearances.

A second plugin is called Avitabs and allows in VR to open any pdf documents, few at the same time… so possible to load real OACI charts, VAC charts, checklists. Even an option to calibrate the imported charts in order to geolocalise the aircraft on it.

Wish something similar will come to FS2020, being a SEP pilot IRL, this is the only way to train for real navs.

1 Like

Agree we need a notepad in VR. Perhaps a way to jot clearances using a lightpen. Not having this kills the training value of the sim for me.

1 Like

Seems that a solution came today!!! :slight_smile:

https://www.rafaelsantos.es/web/index.php

1 Like

Did you get it. Its asking for a subscription to that site to download it. 5 Euro is not that much but the site feels a bit shady as it looks like it has multiple mods and I’m not sure if its the actual mod creator’s site, so who am I paying this subscription, just for someone to host all these files?

Not purchased yet ( I found it yesterday late) but will today.
I wrote an email to the owner of this website and he is developing those plugging (and works in others too).

He told me that this plugin is also compatible with graphic tablet (which is important for me since this is how I wrote my clearances with xplane)

eBag (Electronic Flight Bag) Also works in VR

3 Likes

Any updates on this?

SteamVR now has its own window overlay function. Not sure how is the compatibility would be for MSFS2020.

Option for non-steam MSF2020 users — https://fskneeboard.com/

2 Likes