The camera window in the toolbar has a drone speed slider, on the relevant tab. It starts out at about 4% from memory.
Not as convenient in VR perhaps but that’s what I normally use as I rarely bother to plug in my Xbox controller. I make do with the keyboard for the occasional time I fly around with the drone camera or adjust the cockpit camera. The arrow keys are easy to find by touch 