I have a LOT of addons (thank you flightsim.to) and for addons that are specific to a certain area, like an airport, city, etc, there’s never an issue.
With mods that are global in nature, like We Love VFR, Bijans Seasons, AI plane and boat traffic, etc, those have the highest chance of causing issues. I keep those wide reaching addons in a specific folder in Addons Linker so I can test with them on/off as needed.
But in the last year I don’t think I’ve removed an addon due to performance impacts.
System is middle of the road… Razer laptop, 10875h (8/16 core), 32Gb ram, 2080S maxq, storage are all NVMe.