This happens with all inputs, not just trim. Try binding keys or buttons to the throttle control and you’ll see the exact same behaviour.
Incredibly, the same is even true when using keys or buttons for the stick/yoke. Realise that’s very unlikely (unless someone’s stuck using a keyboard) but it illustrates how little consideration Asobo have given any of this stuff.