Standardized benchmark with publicly uploaded flight file possible?

We, as a community, could create a benchmark for this game. I think…

Someone creates a flight (so route, plane, weather and as as much as possible is the same for everyone), uploads the flt file for anyone to download. Then you let the AI pilot fly the route and record everything with something like CapframeX. Finally you share your results with your settings and hardware so we can create a database.

This can be used for people looking to upgrade hardware, analyzing bottlenecks, tracking down if something is wrong on a specific system, keeping track of changes throughout updates and much more.

Of course this can’t be seen as a perfect solution. There are several problems with this approach. I don’t know in which state the save function is. Changes to LOD after updates or visual up- and downgrades impacting performance are hard to take into account. Add-ons, World Updates, server load and so much more can impact performance.

Does this idea sound viable in any way? Has this been attempted before? Looking forward to hearing your thoughts.

2 Likes

Just use a Bush Trip.

I do not install those.

I had something shorter in mind. Like 5 to 10 minutes

Are bush trips the same from release to release? Do they have the same weather each time? I ask becasue I’ve never even looked at a bush trip.

Again many users do not install busy trips.

While I’m not sure if SU10 did anything to this, I have seen live weather somehow show itself in bush trips, which I can’t imagine is intentional.

I was doing the Nevada bush trip last winter, the one with the Zlin Aviation Savage Cub, one leg a day, and a large cold front winter storm system went through the central California and Nevada area in real life. I went in to fly the leg of the day only to find there to be near zero-visibility white-out conditions with icing, which the Savage Cub is in no way equipped to handle; IIRC I had to abort and wait a couple days to continue.

More recently, I have been working on the Appalachian bush trip on the weekends, and there were a few times before SU10 dropped that there was weather that probably wasn’t VFR (flying through mountainous terrain with thick bands of clouds), but since you’re using the Bonanza G36 with the G1000 NXi you can basically continue even though I’m not sure that the system is actually certified for no-kidding terrain avoidance at low level. I imagine this reflected the real-world weather in that area. After SU10, the weather has looked the same scattered to broken clouds, but I’m not sure if that’s the default weather or if I just got lucky…

I’ve been using a standard flight for my own performance and graphics benchmark for a couple of years. IFR KEWR to KJFK ILS 31L.

There must be something like the benchmarks in X-plane.
Asobo can create such a benchmark.

I think it’s best not to let the accused be the person who chooses the judge.

I use CapFrameX and a record of a deep flight (<500ft) in NY to compare (preset high 3440x1440). But yes, integraded would be better.

So for me:

  • New Update ~22% slower
  • CPU utilization really poor (again). Not like cyberpunk or other AAA games.
  • 2d no problem, but you feel the -22% in VR

I benchmark simulation performance improvements using a stationary position to ensure consistent and repeatable test results for each software update and hardware change. My benchmark is the Diamond Aircraft DA40NG in the KLGB runway 30 takeoff position with clear sky at high noon. Although not the most demanding scenario, it provides sufficient workload for either the CPU or GPU to be a bottleneck depending on sim settings. Included is an excerpt from my benchmark spreadsheet for the latest SU11 sim update.

Interesting, for me the performance is better than pre SU11. Perhaps it effects different graphic tiers differently?

I am using Ultra settings with TLOD=400 @ 5120x1440

@topic: yes a standardized benchmark with several different scenes (airfield, small airport, big airport, country side, city with PG, different airplanes,…) would make sense - built in benchmarks are widely used in AAA games …