Trying to reproduce this. Started in the stock A320 on a runway, no flight plan; takeoff, engage AP with speed / heading / open climb to 2000 ft.
At 2000 ft, I set the FCU to 10000 feet, turned V/S to -2000 and pulled. The plane indeed starts to climb, at 2000 FPM. I then changed to +1500 and pulled again, and it changed the climb rate to 1500 FPM.
So it seems to work as intended when used as intended, but also climb with negative numbers. Not sure how the real plane would behave since the set altitude is higher than the current.
When stabilized I set the FCU to 2000 ft, set V/S to +2000 and pulled; it started a descent at 2000 FPM. Setting to -1500 and pulling again caused a descent at 1500 FPM.
So again it works as intended when used as intended, but also descends with a positive value set.
In other words, it seems to decide the sign based on whether FCU alt > current alt or not, and then use the absolute value of the V/S knob value as the climb/descent rate.