You have to activate the ‘selected’ or ‘managed’ modes with the A320 AP. If using a mouse in the new (not legacy PC only) interface then you change value with left mouse down and WHILE HOLDING LEFT MOUSE BUTTON DOWN click the middle mouse button (for selcted mode) or right mouse button (for managed mode).
You don’t need to use V/S to change altitude. but if you do I think it’s the middle mouse click option to engage it.
To change altitude in managed mode just change the dial to the altitude you want and while holding left mouse button down click right mouse button and a dot will appear in display indicating ‘managed’ mode.
To change altitude in selected mode just change the dial to the altitude you want and while holding left mouse down click centre mouse button (or scroll wheel click).
If you are on PC and in ‘legacy’ mode (changeable in options) the managed/selected modes are engaged by small arrows that appear when you mouse over the dial.