Two things to try.
Clear the DirectX cache: Run Disk Cleanup for your C: drive, clear all checks, check DirectX Shader Cache, select OK, select Delete Files
Run DDU to thoroughly remove all graphics driver remnants. It is available from guru3d.com, where you can read all about using it (definitely read about it before using it!). Used that for my 1070->3080ti upgrade without any problems.
Not sure I completely understand exactly how your problem happens. If it’s the sort of thing that always happens after 20 minutes of things working fine, could be a case cooling issue with your computer, as in your components aren’t getting adequately cooled. Hope all this helps.