Alright, let’s call it annoying.
Maybe you are not used to the convinience all other apps offer where you just move your mouse on them and click.
For me it shows that the devs did not think about such (small) details which (in my opinion) actually make a whole difference when I have to judge between enjoyable vs. annoying.
I simply don’t like this bahvior of the sim / game, I am not used to this. None of my other sims / apps show that behavior.