RollingCache.ccc performance debugging … under SU3

I see, that’s a different view of the difference between streaming and downloading.

Downloaded:

Flight Simulator in C:\XboxGames\Microsoft Flight Simulator 2024

all packages that are permanently stored in individual files on the file system, can be checked for possible updates by MSFS while starting a session and updated on demand

Streaming:

temporary packages for direct use of running simulator and not stored on the file system. they are always up-to-date. some of the Streamed packages can be temporarily stored in the Rolling Cache

that’s the same as downloaded video file vs watching streamed video

1 Like

I think MS / Asobo need to fix these Rolling Cache topics.

Some people say use the cache, some say disable it… confusing

2 Likes

addition: I have not intended any dispute, just I’ve been buffled by your remark about updating streamed packages

I really do appreciate your Rolling Cache analyzes, have read them all with deep interest

Just another Thank You for your analyses. Very nice and informative. –Redeye

2025.09.03 - v1.5.27.0 - Not cached sim content

It is always better to cache locally than to repeatedly download the same data from a remote server … over and over again.

  • The local experience gets faster with cached data.
  • The remote servers are under less stress … which makes them “faster”.

I did try to make my case for “excessive caching” back in 2025-01:

In most cases it is hard to say which data is not cached, because it would require a very intrusive form of project analysis. But in some cases it is very obvious, because it jumps right into the face of a “curious observer” … like a goose.

The following two examples have been around since the launch of FS2024 and are still present in SU3.

Marketplace content

When scrolling the marketplace content with a 500 Mbps fiber-optic internet connection I always see this:

  • Once the images have finally loaded
  • … I scroll down
  • … wait for the images to get loaded
  • … I scroll up
  • … wait for the images to get loaded
  • … and I scroll … and I wait … etc pp

There still is (practically) zero local caching of the marketplace content.

This makes giving more money to MS-Asobo a somewhat frustrating experience. A goose does not really like to sit and wait for the same images to load over and over again. Life is too short for that.

The marketplace UI is most likely based on a JavaScript + HTML UI which gets rendered by the Coherent GT library … which FS2020 and FS2024 are also using for cockpit instruments.

So one can assume that the marketplace images and descriptions are provided by regular HTTP messages.

The online data counter and the “Display FPS” overlay do both show related network activity. We are looking at a very large number of requests … but for content with a fairly small (tiny?) size … so the Mbps level is always very low.

This IMHO to some degree is good news, because:

  • The sim code seems to be actively involved in downloading that content.
    • The Coherent GT engine is not bypassing FS2024.
  • The content (images, thumbnails, descriptions) is mostly immutable
    • … which makes it a perfect candidate for caching.

It should be easy to simply add some local “HTTP proxy cache” layer so that even HTTP content downloaded from JavaScript and the Coherent GT HTML UI subsystem can be cached in the existing RC subsystem.

High altitude Free Flight globe content

When starting a new Free Flight I often go for a spin on the 3D globe … searching for inspirations.

What I did notice back in 2024 still is present in SU3:

  • I spin the globe without stopping, like a mad goose,
  • … and I can watch continuous excessive network downloads
    • … going up to 450 Mbps in the screenshot above.

This happens while all “Live” features are disabled … “Clear skies” only.

Now the strange part is, that once I zoom in “enough”, then sat image data suddenly starts to come from the cache and network activity slows down accordingly.

This is very counterintuitive, as the “space flight altitude” sat images are the most obvious candidates for long term local caching.

Currently my best guess is that I am looking at two different rendering engines:

  • One for the “space flight altitude” showing the round globe
    • … which is not caching its content.
  • One for the actual “game engine altitudes” with a more “flat earth”
    • … which is performing content caching.
  • And somewhere the Free Flight menu view is switching between them … maybe.

To summarize these negative observations:

  • Marketplace data (images, thumbnails, etc.) are still not cached locally
    • … which results in a noticable degradation of the usability of the UI.
  • 3D globe data (textures) at “space flight altitude” are still not cached locally
    • … resulting in unnecessary and excessive network downloads.
    • It is unclear why this happens, because this is a very obvious candidate for caching
      • … due to very high reuse immutable data.

P.S: I hope this answers one of the questions which you, @mjchernis , asked a few days ago.

5 Likes

2025.09.05 - v1.5.27.0 - The mystery of the Online data consumption counter

After I had discovered that Process Monitor is not a reliable source of information for the accumulated inflow of data into the sim and the RC … I started to pay attention to the “Current Data Consumption”, which sadly (also) was not producing reliable numbers for a long time. See for example this thread:

The good news is:

In FS2024 SU3 the counter seems to count correctly.

The feature exists since the initial release of FS2020. Inside the “Online” part of the settings menu one has access to:

  • The “Current Data Consumption
    • … showing the MB or GB measurement of accumulated data downloads.
    • The description claims this to be on a “monthly basis” … really?
  • The “Data Tracking Reset Day” slider
    • … which allows to configure the day of the month where the data counter (should be) reset to zero again.

The sim writes the online data counter once every minute to the local storage into the file …

.. \LocalState\datausage

In theory it would “only” need to track:

  • some file format ID
  • a 64 bit counter for the total downloaded bytes
  • a 64 bit timestamp (or date) for the last recording
    • This could even be extracted from the files meta information in the filesystem.

Which in total would be a maximum of 24 bytes.

However, the actual datausage file is fairly big and requires 269 bytes … a pretty odd uneven and unaligned number.

A quick look at the file paints this picture:

xxd datausage

00000000: 4441 5441 4341 5000 02a5 127f 5f04 0000  DATACAP....._...
00000010: 00a6 0e8b d314 0000 0037 43d7 cc08 0000  .........7C.....
00000020: 00d5 715b 8305 0000 00ff 62d8 3300 0000  ..q[......b.3...
00000030: 0000 0000 0000 0000 0079 0167 ef00 0000  .........y.g....
00000040: 0094 ea79 5401 0000 001d b24d 4904 0000  ...yT......MI...
00000050: 009c 02af f201 0000 00d0 c54b 5a01 0000  ...........KZ...
00000060: 002b 4978 c800 0000 00b3 17ff 1e02 0000  .+Ix............
00000070: 0047 03e2 9c00 0000 000e 7613 b202 0000  .G........v.....
00000080: 0000 0000 0000 0000 0031 bc57 d702 0000  .........1.W....
00000090: 003d 7af3 0a06 0000 001f 7159 4406 0000  .=z.......qYD...
000000a0: 00f6 b6b2 4000 0000 0021 1390 2e0f 0000  ....@....!......
000000b0: 0080 dbaa 3c0f 0000 0027 ac27 f802 0000  ....<....'.'....
000000c0: 00ab b2d2 4d16 0000 0055 d0a9 3c06 0000  ....M....U..<...
000000d0: 00f1 f4d7 7c29 0000 00c7 0379 bb0d 0000  ....|).....y....
000000e0: 001b 5776 7b1e 0000 0084 f6b7 1c00 0000  ..Wv{...........
000000f0: 00c9 583d 0801 0000 005c 94b0 5b0c 0000  ..X=.....\..[...
00000100: 0003 0009 00e9 0700 0241 354c a8         .........A5L.

Some content seems obvious:

  • DATACAP … is the magic file format marker
  • 03 0009 00e9 07 … is the date 03-09-2025

The rest is less obvious at first sight, besides the fact that the columns of “zeros” indicate that there is a form of 8 byte (64 bit) pattern and some periodic structure.

Data usage per day

On 04-09-2024 I deleted the datausage file and the fresh new incarnation looked like this after a short flight:

xxd datausage

00000000: 4441 5441 4341 5000 0200 0000 0000 0000  DATACAP.........
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000020: 00ca a364 af02 0000 0000 0000 0000 0000  ...d............
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000100: 0004 0009 00e9 0701 02a4 60e8 9c         ..........`..

So I would guess that:

  • 00 02 … after the DATACAP … might be a version.

When comparing different snapshots of the datausage file it turns out that only one set of 64 bits in the middle section is changed on each day. So the datausage seems to track usage per “day of the month” … at least that is my guess.

The hex number 0x2AF64A3CA sits in the slot for “Day 4” and would translate to 11532542922 which would mean 10.74 GB of data. This matches to what I did see in the “Online” settings menu at that time.

31 days * 8 bytes is 248 bytes … which fits nicely between the initial file format marker and the date information at the end of the file. It also explains why datausage needs 269 bytes.

However, this all still leaves some questions:

  • Why track data consumption per day?
    • Tracking data volume for different kind of backend systems would seem a lot more interesting … to me
      • … and for tuning the active online features of FS2024.
  • What are the trailing 6 bytes in that file?
    • The first two bytes rarely change.
    • The last 4 bytes look like a CRC32 checksum.

Failing on Reset Day

To make it short:

The “Reset Day” feature has never ever worked for me … not in FS2020 and not in FS2024 SU3.

A fresh new datausage files results in the reset date dropping down to zero.

While the UI slider does show “zero”, the number “zero” cannot be set by actually moving the slider, and there is no documentation what “zero” would mean:

  • zero = No counter reset ever?
    • … clearly a feature that I would like.
  • zero = Always the last day of a month?
    • … because what happens on “Reset Day” 31 … in February?

Calendar dates are a lot of fun … and if you are curious about how much fun that can be I would recommend an old article by Dave DeLong:

However, the deep mysteries of calendar logic cannot explain why in year 6 of the fight simulator the reset feature still does not work … at least it never worked for me … and I really tried to provoke it.

To summarize the datausage aspects:

  • Online data consumption counter now is a correct bytes counter
    • … and it tracks RC data, live weather, marketplace and other downloads.
  • Online data consumption counter does not reset on the configured “reset day”.
    • When the datausage file is first created, the “reset day” is “zero”
      • … but there is no documentation what “zero” … or any other number means.
        • … like “31” in February.
  • The datausage file does store data consumption counters for 31 calendar days.

PS: Teaser … I am still busy collecting “real” RC + FPS insight … and so far it looks like a “mixed bag”.

3 Likes

Yeah, would probably help identify some of the RC Cache optimization opportunities available as well.

2025.09.07 - Searching for the FPS-needle in the RC-haystack

Before I start posting my actual findings I want to provide some background which will be common to all of my current tests. And I want to explain “the ideas” once … upfront.

Contrary to urban consensus, especially when dealing with a complicated blackbox, a goose would say:

Sometimes correlation is the best indicator for causation which we have.

So I am mainly looking out for correlations between all the following:

  • Day of the week … and Time of the day
    • … because CDN and API server stress and reachability will differ.
  • RC size … and fill rate
    • … indicated by the Online -> Data Consumption information.
  • Download speed
    • … based on Debug -> Display FPS information.
  • Rendering performance
    • … based on Debug -> Display FPS information
      • … with a focus on the “MainThread” milliseconds.

Reduce the noise … to get more signal

The more factors can impact the experiment … the harder it becomes to attribute the observation to a potential cause. Especially if the impact is not deterministic (reproducible). So I try to limit “random” impact and …

  • Disable none “deterministic” downloads via:
    • Set “Weather” to “Clear skies”.
    • Set “Air traffic” to “Off”
    • Set “Multiplayer” to “Off”
  • Only use 20 FPS with Vsync
    • … and a 20 FPS frame rate target.
  • Always use the same aircraft
    • … which comes from a local Community installation.

Why the 20 FPS VSync?

FS2024 does offer lots of graphics settings which impact rendering performance … a lot. So it is tricky to pick the “correct” settings.

My high level goals have been:

  • Force FS2024 to download as much data as possible.
    • So I use my full 4K resolution
    • … and almost all graphics settings are set to “Ultra”.
  • Force Ultra-Stutters on RC or download activity
    • … and not lots of micro-stutters.
    • The visual stutter of one or two missed frames should be very easy to see
      • … even without external FPS measurements.
  • Reduce the fan noise
    • … so that I can keep the PC running for long periods while doing other stuff in the same room.

After playing around (too much?) with the different “Graphics” menu options I picked the following settings:

  • Vsync Interval … is One-third refresh rate (33%)
    • This ensures a very precise periodic update pattern.
    • At a forced 20 FPS setting each missed frame is 50 ms
      • … which is very noticable, because at low altitudes the landscapes will “jump” far.
  • Dynamic Settings -> Frame rate target … is 20 FPS
    • This will dynamically reduce the LODs in order to meet the target FPS.
    • The “Terrain LOD factor” in the “Display FPS” overlay will drop below the “green” 1.0
      • … which is a clear indicator that the system (main thread?) could not keep up with the 50 ms time budget.
      • The Display FPS information will record a “yellow” segment.
    • This also does bring down the power consumption of the GPU and CPU
      • … which reduces the fan noise.
      • Without a target FPS the render pipeline still seems to try to produce as many frames as possible
        • … even when the Vsync has already capped the visual output at 20 FPS.

It would be interesting to retry all tests with many Global Rendering Quality settings being “Low” or “Off” in order to compare the rendering impact. But that would take a loooot of time … and this goose is currently too lazy for that.

“Ultra” settings … for “Ultra” stutters

As a sightseeing goose I always fly in “Ultra” everything, as I want my world to look … nice.

I also know that my system (usually) can deal with 4K-Ultra at 20 FPS reliably and quietly.

My system is usually “MainThread” limited but only rarely needs to apply Dynamic Settings in order to stay below 50 ms.

In order to get “Ultra” stutters for these RC tests I want to keep the “MainThread” busy … and as close to the 50 ms limit as possible. But keeping the “MainThread” so busy requires to ramp up all render options as much as possible.

Not hitting the 50 ms (20 FPS) deadline, e.g. due to the impact of slow data downloads, does cause one, two or more frames to not get rendered in time, resulting in a 50 or 100 ms “hang” followed by a visual “jump” of the landscape. A nice “Ultra”-stutter.

“MainThread” milliseconds

In contrast to the FPS indicator the “MainThread” milliseconds are specific for a single frame, and not an average over a longer period (which the FPS is). So I find the millisecond numbers more interesting.

Normally my “MainThread” clocks at 25 to 30 “yellow” milliseconds. Under “normal” stress it does go up to “red” 40 ms.

But really bad spikes will turn “purple” at over 50 … or like here 108 ms:

I still do not really understand which part of the data processing and rendering pipeline of FS2024 is represented by which measurement of the Display FPS debug overlay. But “MainThread” measurements seem the most likely candidate for including RC and server download related “milliseconds”.

The technical nature of that relationship is really hard to confirm. But there is a clear correlation between high download rates and high “MainThread” measurements.

However, when looking at my Display FPS screenshots, keep in mind that in my 20 FPS the colored graph of the “MainThread” has a time span of 3 seconds, while the “Download speed” covers a span of 90 seconds!

As the “MainThread” usually is the place where everything gets done, unless it really has a dedicated thread, it will most likely also include: event handling, memory management, flight model calculations, …

Why the Osprey?

You will see that most of my screenshots do show the Osprey VTOL aircraft.

I am not sure if that is the best choice for these tests … but it is the aircraft that I fly almost all the time, and so I have a good feeling for “what is normal”. My other criteria have been:

  • It must have an autopilot
    • … to allow fairly automated RC stress testing.
  • It must be fast
    • … and 290 ktas (max) for the Osprey does speed up the TIN download and RC fill stress procedures.
  • It must be “low overhead”
    • … and the Osprey model is kind of “old”" and somewhat tuned towards “lower-res”.
      • I find the rendering stress to be fairly low … when compared to other FS2024 aircraft.

I was wondering if the use of WASM might be an unpredictable factor. But in level flight (nacelles at angle zero) the Osprey uses the regular FS2024 flight model, and not its custom WASM VTOL flight model.

So I think the impact of the Osprey on the flight observations is reasonably “deterministic”.

Sweeping TIN regions

For the purpose of ingesting lots of GB into the sim and the RollingCache.ccc file I picked the state of Florida (NA-US-FL) because:

  • Florida … has a lot of scanned high-res “TIN” regions
    • … and those require a lot of texture and 3D mesh downloads.
    • The TIN regions are fairly close together … and mostly near the coast.
  • Florida … is basically flat
    • … and so I can drag the drone cam under my aircraft at low altitude (e.g. 150 m)
      • … and it will not get stuck in one place due to hitting some trees or mountain.

With such a flight pattern I can force the download of ca. 170 GB in 12 hours.

Using recorded test flights … via SkyDolly

In order to perfectly reproduce and repeat a stress test I use SkyDolly as my flight recorder and playback tool.

There is a “native” flight recording features inside FS2024 … “FCR Embedded” … but:

  • I fear that it might impact FS2024 from the inside
    • … as it might be running in JavaScript and the Coherent UI
      • and it might have to deal with very large datasets.
  • I have not found a “loop” playback feature in “FCR Embedded”.

SkyDolly allows me to keep the “playback stress” outside of FS2024 … and I can use its “loop” feature.

Only educated guesses

I tried to make “educated” guesses about how my tools and settings might affect the sim, its data pipeline, and its rendering system. But just to be very open about it:

I might be totally wrong with my assumptions!

At least I hope that this allows others to better understand what I tried to achieve during the tests, which my next postings will be focused on.

To summarize this post about my testing approach:

  • I run my tests without “Live weather” or “Live air traffic data”
    • … so that only “Live (PG/TIN) landscape” is causing downloads during flights.
  • I run in 4K-Ultra everything
    • … to maximize the landscape and TIN data downloads
    • … and to ensure a high rendering stress on the “MainThread”.
  • I render with Vysnc and a 20 FPS (50 ms max) render target
    • … as my system usually is “MainThread” limited
      • … at around 30 to 40 ms with the above settings.
  • … and I “hope” that RC or server data input will cause “Ultra”-stutters
    • … as it forces my system to fail to stay below 50 ms
      … and so it will have to skip one or many “50 ms” frames.
4 Likes

Amazing how you’re approaching this, looking forward to the results!

1 Like

We appreciate your detailed approach - please continue. SU3 has really generated ultra-stutters. I used to be annoyed by the “burp” level stutters that slightly broke the fluidity of flight but now these mega-ultra pauses are ridiculous and at some airports, they can happen at the worst times during takeoff or landing - and some us have very high-end systems that should be able to power through these annoyances, but they can’t. The major question is how we can provide repeatable test conditions to the Devs. This is a very complex sim especially due to the online loading of resources but also because of animations of ground objects and characters and … and a myriad of complex algorithms for LOD issues. We seem to get stabilized in one release and then later break out into stutterville.

1 Like

It’s a shame Asobo doesn’t take advantage of your expertise and willingness to explore this further. I’m sure they have an internal white paper or something that explains the caching process that if they were willing to share with you would answer many of the unknowns and allow you to better focus on and correct the issues we’re all experiencing.

You’d sign an NDA, wouldn’t you?

Thank you all for the positive feedback.

Regarding the question: Would a goose sign an NDA?

No.

I am a goose … how should I hold a pencil?

An NDA or any other contract is already a signal of mistrust. A goose believes in the truth, honour, friendship … and there used to be a time, even among humans, where giving your word was enough.

Life is too short for dealing with companies that demand NDAs … and as an old goose I feel entitled to that opinion.

But on a more practical note, if I would sign an NDA, e.g. to get some secret documents or join some secret alpha release, I would no longer be able to write freely about what I am seeing and thinking.

A goose needs to stay free.

4 Likes

An NDA isn’t ever going to happen, and even tho this is a fantastic thread as was the previous I don’t believe Asobo would ever acknowledge such a discussion, let’s just hope it adds a little food for thought, let’s see what SU4 brings

2025.09.08 - v1.5.27.0 - FPS check during a 22 minute KSJC-KSFO loop

Among the very common questions in the RC context, which also interests me, usually is:

Are the FPS stutters that I see related to the Rolling Cache?

One test which I use in this series of SU3 experiments is …

  • … a recorded 22 minuted loop from KSJC to KSFO … and back.
    • In total it requires ca. 10 GB of landscape data downloads.
      • So it does fit 100% into the smallest (16 GB) RC.
  • I use one fixed (saved) cockpit camera position
    • … so that always the same landscape gets needs “downloading”.
  • I picked a few “marker” locations (times) during the loop
    • … and record their Display FPS info.
  • I did/will run the same loop under different conditions:
    • With a 16 or 256 RC.
    • With an empty or a full RC.
    • Right after launching the sim
      • … and after hours of sim usage.

I picked the San Francisco Bay area because it is a “TIN heavy” landscape which requires a lot of data.

The following results are based on …

  • Test D18
    • Sim launch took 02:25 m:s
      • … in the morning at around 08:45 h:m.
    • RC has 16 GB and is 100% full.
    • The loop test started right after a 12 hour flight,
      • … which downloaded and processed ca. 160 GB landscape data.
        • (Why this matters with a 16 GB RC will be a topic for a future report).

Important: None of the loop related data is in the RC when I start the playback!

Here are the pictures of the first 3 loops at the 09:00 m:s marker of the flight path.

Loop 1 … heavy downloading

Loop 2 … some downloading

Loop 3 … still some downloading

I think the story of those pictures might be easier to see in a table with the relevant numbers:

Test Loop time MainThr Terr. LOD Downloads … over 90 s
D18-L01 21.40 09:00 41.8 ms 0.91 171.56 Mbps very high
D18-L02 22.03 09:00 27.0 ms 0.81 0.00 Mbps some
D18-L03 22.26 09:00 27.9 ms 0.79 0.00 Mbps some
  • D18-L01 21.40
    • … has very high download rates … and a high “MainThread” number.
  • D18-L02 22.03
    • … presently has no downloads … and a < 30 ms “MainThread” time.
    • However, the download graph clearly indicates some serious (red line) downloads during the last 90 seconds.
  • D18-L03 22.26
    • … presently has no downloads … and a < 30 ms “MainThread” time.
    • Again, there are still some serious downloads (red and purple line) during the last 90 seconds.

One question now is, if the difference in the “Terrain LOD factor” plays an important role in the “MainThread” times?

I would say … No … because:

Above is a recording from the previous test (D17-L06) of the same loop at the same location … but this time it has a “Terrain LOD factor” of 1.0 … and a “MainThread” measurement of 29.0 ms. The last 90 seconds show zero downloads.

So FS2024 on my system is fully capable of rendering that location with no dynamic LOD reduction in under 30 ms … no problemo.

Here is some more information about …

  • Test D17
    • Sim launch took 02:12 m:s
      • … in the morning at around 08:45 h:m.
    • RC has 16 GB and is “zero” full (fresh).
      • The launch did download 0.6 GB.
    • Sim downloaded zero GB landscape prior to the loop test.
    • The test performed 34 KSJC-KSFO loops over 11 hours.
      • The first 2 loops downloaded 10.7 GB
      • … and the other 32 loops downloaded only 0.25 GB.
Test Loop time MainThr Terr. LOD Downloads … over 90 s
D17-L01 09.08 09:00 108.8 ms 0.97 201.33 Mbps very high
D17-L06 11.06 09:00 48.6 ms 0.84 0.00 Mbps zero
D17-L20 16.20 09:00 29.0 ms 1.00 0.00 Mbps zero
D17-L34 21.31 09:00 26.6 ms 0.83 0.00 Mbps zero

I already posted the 108 ms (D17-L01) screenshot a few days ago … high downloads = very busy “MainThread”.

But the above picture of D17-L06 raises the more interesting question …

  • Why is the “MainThread” time in D17-L06 so high?
    • … there are zero active downloads!

Here are the Display FPS overlays of all four D17 examples:

The time graph left of the “MainThread” number is the interesting part:

  • The “MainThread” has most of the extreme spikes,
    • … when compared to other render performance graphs.
  • All loops are showing a lot of “yellow”
    • … which indicates not more than 30 ms.
    • This should result in zero stutters.
  • All loops are showing some “red” and “purple” (over 50 ms) lines,
    • … which is where the “Ultra”-stutters come from.
  • The “very high downloads” case D17-L01
    • … has larger “red” and “purple” blocks.
  • The cases D17-L20 and L34 have only few “purple” lines
    • … but they clearly show periodic events … whatever that “really” is,
      • this is what caused my regular “Ultra”-stutters.
    • There are “usually” 5 purple lines per second
      • … so they are spaced with 200 ms
        • … which is 1 in 4 frames in my 20 FPS setup.

The very periodic nature of the “purple” lines is interesting and will need further investigation … especially because it does not exist in all test runs.

I also want to point out, that …

I could have made “purple performance” screenshots during each loop!

… or even loop 1 could have been “yellow” … if only I had pressed the “take screenshot” buttons a few milliseconds earlier. A single number is never a good indicator.

Analysis on Test D17 and D18

So getting back to the initial question …

Are the FPS stutters that I see related to the Rolling Cache?

… and to summarize the … “obvious” … results and correlations of these tests (on my system):

  • FPS stutters
    • … are clearly correlate with “MainThread” performance drops (millisecond spikes).
      • Other parts of the rendering pipeline show a lot more consistent performance.
    • … seem to clearly correlate with data downloads
      • … which is naturally linked to “RC write” operations.
    • … are not visibly correlated to “RC read” operations.
  • Data which is already cached locally (e.g. in the RC, or installed manually)
    • reduces downloads reliably … almost to zero.
  • Even high bandwidth internet connections (e.g. 500 Mbps)
    • … will not be able to download all landscape data during the first encounter.
      • The first loop did receive 80 to 99% of the TIN heavy landscape “just in time”,
        • … so it usually requires multiple loops to actually cache all data.
        • (At least under the following test conditions:
          • 4K-Ultra-TIN … 250 ktas … 2300 ft … 500 Mbps fiber-optic internet)

The key open questions resulting from these tests are:

  • What are those “purple” lines in the “MainThread” graph
    • … and why are they fairly periodic … 1 in 4 frames?
      • They clearly are not “normal” for my test setup.

Next I plan to compare the loop performance differences between 16 GB and 256 GB RC tests.

4 Likes

This does not surprise me - I observed this back in December 2024:

4 Likes

Great post! I’m seeing some unexplained CPU hitches myself recently so this is really interesting.

One thing I’m wondering: If I understood you correctly, you are using the “dynamic FPS” / LOD setting for those tests? Doesn’t this introduce an additional variable that’s not really needed and might overcomplicate things?

True.

But my idea is, that I want to make “poor performance” to become “easy to see” … and I want stutters to be an “unusual” and not a “constant” feature.

  • Not meeting the FPS target
    • … will first reduce the “Terrain LOD factor”
  • … and that will leave an easy to see “yellow” bar in the “Terrain LOD factor”.

I think that what I did see in Test D17 and D18 is that

  • … the stutters are clearly not related to the “Terrain LOD factor”.
  • “Terrain” and “Object LOD factor” are adjusted gradually and more slowly.
  • “Terrain LOD factor” is a reaction to being “Main thread limited”.
  • “Object LOD factor” is a (mainly) reaction to being “GPU limited”.

As I always point out, I have no clue if my approach really makes sense. Time will tell.

I currently try to collect as many recordings as possible with identical settings because I am mainly interested in the RC and data download impact on the rendering.

But @BearsAreCool510 you are right … at some point I should start to play with the different settings and see what will change in the rendering results.

Obviously I agree with you @SmotheryVase665 , that download performance of FS2024 in 2024 was … a fiasco.

I also provided lots of evidence in the previous thread … for example here …

… where I had to wait 2 minutes for a TIN location to load fully.

However, the performance today is orders of magitude better than back in 2024.

To underpin my argument let me add the following numbers for the first three D17 loops:

Test Download Volume … in %
D17-L01 09.00 10.07 GB 99 %
D17-L02 09.23 0.05 GB 1 %
D17-L03 09.45 0.00 GB 0%

In D18 it looked like this:

Test Download Volume … in %
D18-L01 21.31 8.19 GB 81 %
D18-L02 21.54 1.95 GB 9 %
D18-L03 22.17 0.00 GB 0 %

For a reasonably low and fast flight (250 ktas, 2300 ft) a download success rate of 80 to 99% during the first loop is pretty good IMHO.

Looking at the screenshots I cannot really tell the difference between the three loops.

But thank you for making me realize, that the bullet item in the original post may sound too negative. I will try to rewrite it with a more appropriate wording.

If this is what “all” people are seeing in SU4 (beta1) …

… then maybe I should stop searching for that FPS-needle in the RC-haystack … because it …

Looks like Asobo found … lots of FPS-needles.

2025.09.10 - v1.5.27.0 - The FPS KSJC-KSFO loop with 16 and 256 GB RC size

Another very common question in the RC context since the launch of FS2024 (or even FS2020) is:

Does a larger RC size have a positive or negative impact on the FPS?

In early 2025 I had the feeling that a larger RC size (due to frequent RC index writes) did make stutters more likely. But back then I did not investigate that in any depth on the Display FPS side of the story.

For SU3 I want to have a form of “baseline” … so that I can later compare those numbers with SU4, which in Beta 1 seems to have very noticeable improvements for many users.

I have already explained the background of the “KSJC-KSFO 22 minute loop” test in a previous post, and I also provided the background of the D17 and D18 test flights. I will not repeat the details again (you can scroll up if like) … but I will provide a more useful version of the Display FPS overlays screenshots:

I also have some new screenshots from test D26 and D27 which both did use a 256 GB RC:

First allow me to explain what the text items in the header boxes mean:

  • RC, fresh
    • … means that prior to the launch of the sim the old RC was deleted.
      • So there was no old (cached) content in the RC.
      • For D17 and D18 this meant, that all loop landscape data needed to get downloaded.
  • RC, 100% full
    • … means that during the usage of the RC file it reached the “100% full state” at some point.
      • This could have happened prior to the test (sim launch) which is shown.
    • In all examples shown above this also means, that the RC already contained all loop landscape data.
  • GB (total pre-this-loop) RC downloads
    • How many GB in total did all the sim launches, which have written data into the RC, download from the servers prior to this test.
      • Obviously, for example in the 16 GB cases this means that only the “least recently used” 16 GB of 175 “GB pre-loop downloads” are still present (cached).
  • pre-this-loop flight time
    • How many hours did I fly during this launch of the sim
      • … prior to the KSJC-KSFO loop that is shown in the screenshot.
      • So that does include previous loops.
  • loops started at … UTC
    • The UTC time of the day when I started loop 1.
      • Maybe this can help to identify “busy server” periods.

But what is that white box in the “MainThread” graph?

In contrast to e.g. the “GPU” measurements the “MainThread” does not show the [min; max] med information. So the only idea I have how to “somehow” make differences easier to see is to add a clear visual “base line” (white box).

But there seem to be no major differences in the “RC read” cases, even when D26-L24 looks somewhat more stressed on average.

Analysis of the 16 GB vs 256 GB RC numbers

Reading the visual tea leafs I would summarize the observations:

  • The FPS performance does not degrade (noticeably) while flying the same loop for 10+ hours.
  • “Ultra”-stutters
    • There is no big difference when it comes to the “purple” spikes.
      • The “1 in 4” frames pattern is always there
        • … at the time 09:00 m:s location of the test loop.
  • “RC read” performance
    • There is no big difference when comparing the 16 GB vs 256 GB RC.
      • D17-L20 looks like D26-L24 looks like D27-L02.
  • “RC write” performance … in the context of downloads.
    • I mentioned in a previous post here, that “Ultra”-stutters clearly correlate with data downloads
    • … but in SU3 there is no big difference when comparing the 16 and 256 GB RC.
      • The related screenshot of D26-L01 is similar to D17-L01 … but not included in the screenshots above.
  • A 256 GB RC clearly will have a higher cache hit rate than a 16 GB RC
    • … and so larger sizes are better, as they should reduce “stuttery” data downloads.
  • In SU3, so far, there seems to be no obvious FPS penalty for using a larger RC.

To put it all in one big sentence:

In SU3 I would recommend to use the largest RC size possible … but even in SU3 sizes above 256 GB will only waste storage space, due to the fixed size of the RC index table.

3 Likes