You do have to consider all measured latencies. It confuses a lot of people. This is not a “lower graphics, get MOAR FPS” environment that a lot of folks are used to. Simulators work differently.
Have you seen the full thread detailing graphics settings and their relationship to hardware?
It is a lot to get through, but if you read the entire thing, it should help you understand. Some of the settings have changed due to sim updates, but the basic concept of tuning using render latencies has not.