So I decided to abandon this setup. Apart from the annoying Radeon app eating up CPU at startup, the GPU temps were also not great. Ended up connecting all the monitors through DisplayPort to the 3080. Also fixed the airflow so that more air flow to the GPU.