If anyone is having these kind of issues, then try Lorby Axis and Oh’s. Remove all assignments to the rudder AND brakes in MSFS controller screen.
Now assign the rudder and brakes with AAO instead of MSFS. All your other assignments on the yoke and throttles will still be done in MSFS if you want that to be so. But put your rudder and brake assignments in AAO.
The above will definitely cure your problem unless (very unlikely) the rudder hardware is bad.