What you are talking about explains performance problems in general very well, but not performance degradation. Assuming the “data bubble” around the aircraft is always the same size, if the system could not handle the data load for the selected level of detail, it should be slow from the beginning to the end of the flight, only dependant on the complexity of the terrain.
However, people experiencing degradation have splendid performance at first and get down to single digit frames per second as the flight progresses, no matter the terrain.
Memory does not have to be “cleaned out”. When you delete data on your computer, no matter where it simply gets flagged as free space and is overwritten. It’s not actually being cleaned out. A write operation thus is always more costly than a delete, since a delete does not actually do that much.
The problem is, that for some reason the simulator does not release some of its data at all. Not even after you quit the program. There are simply no or not enough mechanisms in place that help the simulator to alleviate memory usage.
Memory usage should vary with the complexity of the terrain and to some extent it does, but the committed memory is not freed up properly as it should and this leads to degradation at first and CTDs second, depending on the VAS available.
People flying for longer periods of time, either long haul or just slow VFR for hours, will be more affected than people who play maybe 1-2 hours.
PS
I have not yet tested it in-depth, but I think the new patch improved memory handling to some degree.