I’ve seen this behavior, too. What is extra annoying is even when I have my TFRP pedals hooked up, and the twist grip rudder function locked via the locking screw, it still would see small rudder inputs as you describe.
What I did to alleviate it was to increase the rudder axis deadzone. Give that a try (if you haven’t already) and see if it helps.