I have the same card from Zotac and an i9-11900K. I am running the sim mostly on ultra (where it makes sense) and the rest on high. My Lods are set to 4.0 in the usercfg.opt.
My Ram used in the video card is nearly always at 8-9 GB . In the sim I never encountered any problems ,like stutters(actually, I never have stutters) or long scenery loading time. I am using DX11 and TAA . It might be a defect in your card.
Scott