Well, I did a thing… 
I originally wanted the X but couldn’t get my hands on one a couple months ago without paying scalper prices. I’ve been pretty happy with the S but, based on some of the head to head comparisons out there, I wanted to try the X anyway.
So far the comparisons online have been pretty spot on as far as the richer detail vs a little more stuttering in challenging scenery such as London, England. I’ve only had a couple of hours on the X but I think it’s a keeper.
That’s not to say people should avoid the S. Far from it. At 1440p, 120Hz and VRR the XBox S is buttery smooth with almost no stutters even in dense scenery. From a strictly smooth “flying” point of view I think the S wins.
An observation I have so far which I’ll pass on is the graphics settings matter a lot more on the X. I was originally thinking the way to go would to be use the same settings as I’d been using on the S: 1440p / 120 Hz / VRR enabled but the stutters were horrible by comparison with this setting on the X. My monitor is native 1440p but it does support 4K at 60Hz. When I reconfigured the X to use 4K / 60Hz / no VRR things immediately settled down and, although more frequent than the S, were much closer to my experiences so far with the S.
It seems counter intuitive to me that higher resolution should result in smoother game play but it does on the X. Total speculation but perhaps it comes from the extra cycles of up-converting / down-converting the 4K textures / screens to 1440p? 
I like looking at the scenery a lot, mostly fly fairly low and the farther out detail distance makes it more immersive for me. If I was an Airliner enthusiast and usually flew at high altitude I don’t think I’d see any reason to get the X over the S. (The only game I’m interested in is MSFS so there may be other reasons to choose an X over an S - for example, the larger internal SSD and 4K blueray player but those aren’t considerations for me.)