You’re absolutely correct, but the problem is not simply the effectiveness of controls, but the feedback imparted by the airflow. During real-life slow flight, the controls are less effective, but also less stiff, leading to what we all call “mushiness.”
Conversely, when cruising, the controls have a lot more authority, but are generally stiffer due to the dynamic pressure and stability forces acting upon the control surfaces.
The sim does in fact account for the loss of authority - try bringing in a Cessna at 80 knots versus 50 and see how much elevator authority you have in the flare. What it’s not giving you is the feedback or even the re-centering effect that is imparted by trim at various airspeeds.
And forget the effect of stall buffeting or turbulence, haha.
No two airplanes fly exactly the same in real life (especially when going between makes and models). So I look at the sim, and each aircraft within it, as just another airplane to learn, with all the weird quirks. Trying to compare apples to apples doesn’t work.