CAT III ILS fine until landing

A320 V2 Cat III ILS goes fine until I touch any control after touchdown. Then the aircraft veers to the right into the grass. Any idea what’s happening and how to prevent this?