Assuming it’s a wired controller is it possible that the USB cable or the connections inside from the cable are worn?
I had a similar issue but in my case it was clear to me that it was caused by a worn strain relief and the cable going into the controller housing was not making good contact inside the controller.
I was able to open up the controller and improve the connection for awhile but eventually it simply was way too spotty to continue using. I replaced it with a $20 Logitech controller and all is well again.