That’s what it sounds like to me based on what you’re describing. The yoke in the cockpit will prioritize in following the AP when it’s active over what you’re inputting with your physical yoke. And it should be doing the opposite of whatever you’re doing to try to keep the plane on course. Until you fight it too much and AP disengages. lol