Reading trough the Beta-Forum we see different results when looking at performance, which is normal, of course. Everyone tests on a different system, in a different scenario. Some talk about performance improvement, some about regression. Some talk about DX12, some about DX11, some compare Native SU9 with DLSS in SU10… in short, it is a total mess and anything than meaningful.
Not even 1% of the users reporting about performance are really providing any evidence (like comparisons with SU9) of how they are testing and i tend to believe that a mojority of them dont even use a proper way of testing for performance. It seems many are just going for the “feel” or “stutters”.
So, i would vote for a benchmark-scenario that any user can use, like many games do in the graphics menu. This would be much better to have an actual coparison about improvements/regression after updates.
An easy way to make a benchmark is to pick one of the discovery flights because they’re always the same plane, same location, same weather, same time of day, etc. Then just record it with CapFrameX and compare.
Thats cool, but 99% of users dont. And they dont do proper performance testing. Most dont even use an FPS counter or have an understanding of how to create a meaningful testing-scenario.
Some fly a Cub in the deserts and talk about great performance, others are testing in a mainthread-limited environment, some in a GPU limited environment. We need one meaningfful baseline that covers different typical scenarios that everyone can use and share the results. Only then, we have a baseline when we are reporting performance improvement or regression.
Right now, the performance threads are a ■■■■■■■■■■■ and anything else than meaningful and can give the developers a completely wrong indication.
Essentially “timedemo 1” for Quake, if anyone remembers that.
The demos would ideally fix the graphics configuration to known defaults, a bit like the demos for X-Plane, which override whatever you have configured yourself. That would ensure accurate comparisons could be drawn.
I already commented something similar to OP yesterday, so voted for it. Betas should come with a set of test cases that all users can run using the same settings and the same enviromental conditions (weather, traffic and location). Is the only way to have a real comparable record of the results among different users/systems.
Same here … I see folk with 5xxx cpu’s, and 3080’s with worse performance than I have even when my only OC is PBO enabled. My spec really is nothing special except for CL14@3200 and a mesh case. 3800X, rtx3060(12GB) and stock cooling that never gets hot - Like you say people don’t tune MSFS properly and just settle on mediocre.
Make your own benchmark. Use replay tool in experimental options. I have made one and will record using default graphics settings and then compare with next version using same time and year and also rain preset.
Added Min, avg and max on the photos. Interesting both had same avg and clear skies had lower minimum than rain.
Even if we have an official benchmark there is so many different locations in the world that may be unoptimized. Better to create an own where you usually fly and then optimize your settings for that place. I’m sure you will not fly where Asobo choose to make a benchmark. The settings you choose will only be optimized for that official benchmark location.
Here is my benchmark i will use to compare if performance has increased every update from now Still processing into better quality. First benchmark starts at 1:58. second starts at 12:52
Sorry for bad flying, first time i was flying hornet. Wanted to choose a fast plane for benchamrk.
My next benchmark will be Ultra preset with t-lod 400.
I could share the replayfile but i can’t attach that filetype here and it’s not official
Will make one more replay with only camera movement on ground.
I like the idea of a standardized benchmark. However please note that in VR these overlays (RivaTuner?) do not work. It’s difficult to view performance statistics during flight in VR. I use CapFrameX to make a 1 minute capture and view the analysis afterwards.
Anyway, the point is that if somebody creates a standardized performance test, it would also be needed to standardize the capture method and the statistics that are reported along with game settings etc.
Well, Asobo has that FPS counter in dev mode. I’m sure they can record benchmark log of that if they want to.
Besides if you make a benchmarktest in VR it’s running by it self. Why do you need to use the VR while it running?
I’m not complaining about performance though. I think the performance for this sim is really good. I can tune the settings depending on where i’m flying. I can run it in 60FPS if i want to with my system. I bet those with better hardware can run it with higher settings than me.
Optimised settings for KLAX area rain (know i will have 60FPS mostly everywhere with this setting) If i use complex airliners i may need to accept lower FPS:
Highes (VSYNC ON)=61FPS
Not much of a benchmark if you are essentially removing vital sim systems like weather and planes. What if there was a problem with the weather implementation? It wouldn’t show up in your benchmarks making the whole thing worthless. It is one of those things that sounds like a good idea until you think it through.
Other games do have built in benchmarks but they also tend to be unrepresentative of actual gameplay as any conscientious benchmarker will tell you. They will freely admit the only time they resort to using them is in, for example, multiplayer which has an element of dynamism which makes it difficult to replicate scenarios. You may say - but…but…that’s why I suggested removing weather and planes. Well imagine creating a benchmark for a multiplayer shooter that removed soldiers and explosions. What would that tell you?
Anyone can benchmark their own system (and it is probably the best way) but it will only tell them about their own system and nothing else. I can’t explain why I get better frames with a 3070 than some people on here with 3090’s but, if the frame numbers they are posting here arent waffle, then I am. Time spent focusing on the causes of that, I suggest, would be far more productive.
They could create a benchmark just to appease the forum but I really struggle to see the actual benefit.