UPDATE: Shortly after posting I noticed Nvidia broadcast would show massive gpu usage spikes at times. I have uninstalled it and have not experienced these frame drops since. I would use broadcast to cancel out background noise since I fly on speakers and talk to atc through a headset, but since Nvidia broadcast uses your gpu to do so, it caused massive gpu usage spikes which I think was the issue.