I would call the current streaming model a partial success at best.
Jorg did say they were looking into making aircraft downloadable (SU2?) which will be a huge improvement. Hopefully Xbox users will have enough local storage to make that a reality for both platforms.
I know that scenery is a whole other thing due to the massive (petabyte) size of the world’s data set. Hopefully the RC can be used to cache scenery more efficiently as well.
OR they could release two completely different software…oh, never mind.
I buy it completely because from a business perspective, it is obviously best not to exceed the storage capability of your average customer. While hard-core simmers wouldn’t bat an eye at the need to buy more SSD space for $160 or more, that is likely not the case for the majority of MSFS users (especially on XBOX).
That’s a fair point, and we don’t literally need an extra 2TB.
I already have space to download favourite airports and planes - so why not make it permanent? FS20 is also installed, so I could remove that to gain space.
AI yes. It’s easy to (automatically) download plane, start and destination airports based on the flight plan.
A to save permanently and a 1, 5 or 10 mile range/minimum scenery height option would be complete.
The cache can still function for occasional detail. To clear up space we could delete favourite airports and planes in My Library. Simple.
Yesterday I made a flight from LPFR FARO LEMD MADRID BARAJAS with the 737-8 max and I didn’t have any problems, only when making the trip to the gate when I landed, don’t jerk, it was raining cats and dogs and it was an incredible flight, I don’t understand why sometimes it goes well and sometimes the fps drops brutally and it jerks, yesterday was good configuration live traffic, real-time weather and 100 GB cache memory paltaform Xbox series x
Regarding the cost of storage, I have two 1TB hard disks sitting in a drawer unused. They were replaced by SSDS over the years.
So I could add 2TB of extra storage for free and I won’t be the only person with spare hard disks.
My records show 100 to 157MB/s speed read access, at least 10 times a streamed speed and a thousand times faster than launch day!.
The only platform that genuinely benifits from streaming are old Xbox’s with limited memory, which will get scrapped for newer models eventually.
I think the argument for downloaded textures is stronger than ever. Maybe the 80:20 rule, 80% downloaded and 20% streamed with latest updates, that can be permanently stored in My Library if the user chooses.
You are missing my point. This is about what the average MSFS user has or is willing to buy, not you or me. Also not the majority of people on this forum but the silent majority out there that is balancing the books.
Given that Asobo said that data processing (caching) updates will be a focus in Sim Update 2 … and given that the beta period already has started … I decided to wait with further analysis and comments until the final Sim Update 2 is out.
About 2 month ago I posted my ideas for “easy to implement improvements” …
I think the summary of that old post still makes sense to me … even with the new insights which I gained after that.
A “Hotfix” cache update should respect key constraints:
Any local caching solution must be able to work at the 16 GB default cache size.
No changes to the LRU Cache code base … or the present CCC file structure.
A “Hotfix” cache should try to achieve the following key goals:
Increase the cache Read:Write ratio … significantly.
Avoid increased cache item lookup latency.
Avoid too many necessary code change in the sim.
This could be achieved by splitting todays RC file into three cache files for different content categories:
The list of improvements is truly impressive … but related to the Rolling Cache I do not see any direct reference.
The only items which I would consider “somewhat” in a general “data processing pipeline” category are:
Stability & Performance
Reduced latency with HTTP requests
Streaming
Moved StreamedPackages to installation path next to Community and Official folders
So I guess the next iterations of Sim Update 2 will have to show if and what changes will come to the Rolling Cache subsystem.
I am still not completely sure if Asobo actually “owns” the code which operates the data cache. Most (all?) data seems to get downloaded from Microsoft Azure, and Azure provides a CDN layer and local caching solutions. Perhaps Asobo is sitting on top of the Azure Blob Storage stack and has no “easy” way of modifying the Azure caching algorithms to really fit the FS2024 usage patterns.
We can speculate forever on the cache, as there are no metrics reported.
Percent full would be very useful, as would the percent split between installed data and newly streamed data.
I’m afraid that streaming is the Achilles heal of FS24. Life would be smoother with installed airports and planes.
It’s not rocket science to scan the Log Book and build a list of favourite airports and planes, with an Install Now button
That action could extract assets from the cache.
HighReuse is a good concept. I like it! Kind of obvious as we tend to have favourite planes and airports.
Although to me HighReuse simply means install permanently. End of problems.
I would go as far as storing HighReuse textures uncompressed, for the fastest load times.
It would be interesting to see the actual use cases, e.g. mean number of airports and planes per user.
I won’t use 30,000 in my lifetime, probably never more than 30 as a rolling average. Planes maybe 4.
Interesting and disappointing if true. I’ve had a very frustrating time with the rolling cache, such as it redownloading runway textures at my home field I fly out of every time I use the sim, or it redownloading gauges after switching the camera, despite having excessive system and video memory and rolling cache size. These flaws are made more painfully obvious as I don’t have the fastest internet either. That they’re using a generic, off the shelf solution they don’t have control over would explain some of this behavior. Augmenting it with local installs as others suggested sounds like the way to go.
Today the 4th beta of SU2 was released … and there is still no item which clearly references “changes to the caching and streaming” pipeline.
I doubt that SU2 will see any major changes, because I would not expect Asobo to release such fundamental changes without an extended beta test period.
To some degree that would be in line with the last dev stream, where the team mentioned that asset downloading was pushed to SU3.
It still would be “nice” to see such changes in SU2, but at this point I am no longer holding my breath.
For performance reasons should the RollingCache.ccc file be saved on the same drive as Packages (i.e. Community folder, Stream Packages, etc.) as opposed to in the LocalCache folder on C: while everything else is saved on D:?
IMHO it does not matter … as long as you have enough space on that drive.
Disk access speeds do not seem to be the real issues with the RC. From what I have seen I would say the big issues are:
The RC is not good a caching important data … ergo
… too much unnecessary very slow network access to often overloaded backend systems.
The internal data pipeline seems to use a synchronous access technique to read/write the RC
… and this is where unnecessary big index writes into the RAM (not the SSD) are blocking other RC operations.
In general, due to FASTIO, all accessed RC data is cached in RAM. So none of the problems that I listed above will go away if you pick a drive that is 10 or 100% faster.
This was the post where I did a deep dive into the FASTIO aspect:
I hope @themman92 you find this feedback somewhat useful.
As I wrote in other posts, from what I have seen there is currently no really good solution. I personally am using a 256 GB RC in order to reduce the network access … but my measurements show that this very likely is causing some of the FPS stutter that I see.
I guess SU3 is what we have to hope for in this regard.
To me the Rolling Cache negates the whole point of streaming the sim.
What is the point of streaming to save on installation if you have to download it to disk anyway during play to get a good experience?
I think Asobo really need to look at how we balance streaming and installation.
Sure with streaming you only need install the base and then stream in what you use and not everything, I get that but it comes with drawbacks of needing very stable connections and fast download speeds and the servers to be functioning 100% all the time and where RC is needed we end up still needing masses of drive space.
It think MSFS 2024 went much too far with the streaming aspect, so much so that it seems more like a Microsoft logistics experiment/test than a practical solution.
What we need is the ability to download the most streamed files and any data that is consistently replicated when flying such as aircraft (which we now have I think)and base scenery data etc that is non specific to a particular region.
Things like obstacle placement data, ships, road traffic, tree data etc etc and associated models/textures should all be downloadable IMO and removed from streaming data.
@nenenui this thread is amazing. Albeit it’s above my head at points but your insight is extremely thought out and throughout. Quick question - What software are you using that’s allowing you to see how the read/write of data is occurring within the rolling cache?
After some time I noticed that Process Monitor does not really “record” the process activity but it “only” takes samples of the activity. So the recording is not 100% of all activity … but it feels like 80 to 90% of all file access gets recorded.
(OK … I should have read the documentation of Process Monitor … because this is clearly spelled out there … so I guess a lazy goose must discover it the hard way)
Process Monitor also allows a CSV export of the recordings which I then “post-process” … with find-replace-grep-sort-count etc.
to save permanently and a 1, 5 or 10 mile range/minimum scenery height option would be complete.