So it turns out hand flying as I mentioned in my post above makes no difference.
But what seems to be consistent is that if it’s not following its GPS path exactly during the flight, that same offset will be applied to the localizer path on the way down the glideslope and it will miss the runway.
I can’t deliberately replicate it unfortunately