“Study level” and “tested by real world pilots” are two things that have driven me crazy absolutely forever in simming. I’ve previously been tempted to start a thread with a rant about both and perhaps this is as good a spot as any.
Background.
I come with many thousands of hours of flight time over 20 years of real world flying experience for the most part in an emergency response role, in all phases of flight, in all situations, IFR and VFR, across many aircraft types, in various locations around the world. I also come from 40 years of simming experience as a user, consultant, and modder.
———————————————————–
First, “Study Level”. There’s no such thing. Never has been. Likely never will be.
Whatever you (as the user) have as a definition of “study level”, the definition is in the name. But regardless, it has been strongly implied over the years that “study level” would mean an aircraft that can be used by real world students/pilots on type to use it to prepare for training or a ride. It has since been muddied by the term being thrown around by various devs who add in functional circuit breakers and call it a day, as well as influencers throwing the term around like candy, but I digress.
Can one, as a student/pilot of the specific aircraft IRL use the sim to become intimately familiar with, and therefore capable of “studying” to the point of passing a ride, the following
- all aircraft systems, no matter how big, small, or nuanced
- all systems failures and/or emergency situations
- how a specific aircraft handles in all phases of flight
- (perhaps most importantly) how an aircraft feels in all phases of flight
#1 and 2 it could be argued are theoretically possible but I’ve yet to fire up a sim aircraft that satisified both fully. In some study level/complete/100% accurate/etc aircraft I’ve fired up, I can find fault with these two things within minutes with zero knowledge of the real world aircraft. Allow me to open up an AFM and I guarantee I’ll find an abnormal/emergency item that the “study level” aircraft does not simulate. That’s outside of particular non-AFM system nuances that can only come with extensive time on type. In particular, some of the hugely popular “study level” aircraft such as PMDG are very, very far from being that detailed. “Study level” they ain’t.
#3 is also theoretically possible with enough work on the flight model and engine parameters within knowledge/programming/sim limitations. I’ve yet to fly an aircraft in the sim which doesn’t have a compromise in one area or another and by pure definition of “sim limitation”, some things such as icing impact (just one example) just simply cannot be or are not simulated to a “study level” degree.
#4 is impossible. Quite simply impossible. With the variety of control setups out there, what might “feel” like the real aircraft (excluding things like physical turbulence, G-effect, specific airframe nuances, etc) to one person absolutely will not to another. Compromises come in again, undermining the core definition of “study level”.
Myself, I prefer the term “high fidelity”. OK, you put in circuit breakers and some basic failures? Yeah, that’s high fidelity. I embrace that term. Black Square, A2A, etc are all high fidelity.
———————————————————–
Second, “tested by real world pilots”.
As mentioned above, an absolute key to this is having real world pilots that not only have extensive time on type but having extensive sim experience, ideally not just in a user but in a background capacity.
Without the sim knowledge, consulting pilots are told “X isn’t possible due to Y in the sim” and they have no choice but to believe it. Even with sim knowledge, real world pilots are more inclined to believe what a dev says is a “sim limitation”, even if it solidly is not. I’ve seen it repeated here in the forums ad nauseam. And oh boy - the number of “sim limitations” I’ve gotten around with a simple, non-invasive mod I simply cannot count. Honestly artificial “sim limitations” could be a whole other topic but that’s outside the scope here.
Let’s talk about the wrong pilot experience for consulting on a sim aircraft. Here are some very specific aircraft behaviours in some very specific aircraft that I have asked devs about and they have verified that real world pilots have backed up:
Note that these don’t all apply to one dev/aircraft but just come from years of experience with various
- When you lower the flaps, the aircraft loses lift and drag, dropping and going faster
- When you lower the flaps, the aircraft has an uncontrollable nose up/down movement. “Just ride it out”
- When you lower the gear, the aircraft has an uncontrollable nose down movement
- When you lower the gear, you won’t be able to maintain altitude
- The aircraft won’t stay running at idle
- The aircraft won’t stay running if you lean on the ground
- The aircraft likes to idle high
- It’s a high powered prop/turboprop/jet/whatever and so it taxis fast. Just ride the brakes
- The prop has no drag IRL
- The prop has excessive amounts of drag IRL
- The aircraft doesn’t actually handle the crosswind the book says
- The aircraft auto-levels due to dihedral
- This aircraft is so high powered it basically doesn’t have a service ceiling
- This aircraft would never be able to reach the book service ceiling
And so on. Those of you with a keen eye will see that a vast majority of these sorts of things come from consulting with a pilot that is either very inexperienced, very inexperienced on type, or just completely misunderstanding what is happening in a specific phase of flight. Yes, this could be a lack of connection of the real world behaviour to what the sim is doing as well. This could also potentially be a communication problem between dev and pilot, miscommunicating what is happening in the sim vs what is happening IRL.
This is outside of the average pilot’s tendency to exaggerate. What I mean by that is if you have a pilot of a specific aircraft type that is said to have X behaviour, he’ll tend to play it up like it’s a huge deal even if it’s not. I’m not immune to this. Ask me about flying something like a Metroliner and I’ll tell you how it’s a negatively-stable beast just trying to deviate from level flight. Ignoring the fact that it’s been 15 years since I flew one, if it were as bad as my simple statement about it, it would have never been certified. But the wrong dev might take that statement and run with it, making a version that is truly negatively stable and very difficult to fly. At some point in sim world we’ve come to equate “difficult” with “realistic” and I have no idea where that came from other than this tendency for pilots to exaggerate minor negative characteristics. Add into the equation that that characteristic flaw IRL probably doesn’t come across in-sim due to item #4 above, and the inclination might be to crank it up.
Now, even if you get the absolutely ideal pilot consultant for your new aircraft project, what is their setup? Are they in VR? Are they using an xbox controller? Do they have sim realism settings high or low? Do they have a $10000 full-motion setup in their basement? Are they playing on a laptop? Do they have curves in their controls? And the list goes on and on.
Personally I take “tested by real world pilots” as a bit of a caution as some of the most shockingly wrong aircraft behaviours I’ve seen in the sim have come after “real world pilot” testing. And I say that as one of the “tested by real world pilots” pilots for many past, present, and future sim aircraft out there. I’m not infallible and neither are they.
—————————————————–
All of that being said, don’t take that as hate towards any specific aircraft or dev as I enjoy a wide range of capabilities in the sim but I don’t do it with the false knowledge that I’m somehow flying a 100% accurate, true representation of the real thing. I understand that I’ll probably want to tweak or “live with” a few behaviours sim aircraft have and that might not just be a simple sim vs IRL limitation but in some cases it’s the charm of it too. We keep inching our way towards true IRL representation in sims. In some nuances it’s a long ways off. In others, it’s much closer. It’s not perfect and it never will be but I do love watching and being part of the effort even if I do have to look away from a few things from time to time.
It reminds me of a very old joke. A man goes to the doctor and says “doc, it hurts when I swing my arm above my head” and the doctor says “well just don’t do that”