Same issue on xbox series x.
Peripherals in my case: (xbox controller wired + mouse wired + keyboard wired + Thrustmaster Hotas One) through an USB hub to xbox.
A/T flick happens very often, especially sometimes xbox controller start to vibrate to waring stall whilst AP is turned on.
Have tried reset auto thrust switch (upper the A/T button), shutdown/restart engines, flight level change reset, but no luck to finger out the logic.