So good news: I got in touch with the aws rep for one of my employers, explained there were troubles using Polly within windows applications, using p2atc as an example together with a development reliability question, and they escalated it up to their polly devs.
Hopefully we can see some reliability improvements in 2022!