Autopilot problem

I’m assuming you are using a joystick. Ensure you have a dead zone set as this seems to be a common issue especially with the A320. I have seen at least one other post resolved by this.