Should we be able to vote against wishlist items?

No, in a democracy you have a choice between different options, not one option or not that option.