It doesn’t matter in any real aircraft. I’ll have to test that ASAP.
I wouldn’t be surprised if this is another bug. 
If you e.g. change the V/S you need to pull the V/S again to confirm this change in the MSFS A320, but that’s not the case in the real A320.