Are you trimming your aircraft correctly?
I assume that the Airbus stick is identical to my T16000M and the forces are already nicely low on that one.
With even lower forces it would be impossible to realistically ‘trim away’ off-center forces.
If the aircraft is always in trim, fatigue shouldn’t be an issue.