This is almost mentioned everywhere and in every “get more FPS” video on YT to disable the gamebar and ESPECIALLY the background recording to get better performance. If you have this enabled, Gamebar will record your gameplay in the background which might cause performance-impact.
If you guys are at it, also disable the Shadowplay replay function, as it does the same, constantly recording in the background. However, i have that on and on my system it does not have an impact.