Which looks a lot like a xwnd issue and even just a few knots will cause weathervaning.
Xwnd doesn’t exist once you are in the air.
It occurs with the default A320, but not with the FBW A320?
It would be nice to know if the FBW guys have modified the yaw behavior of their A320.