I had this issue too. To fix it, use the Xbox controller with a bluetooth connection instead of a wired connection.