Hi Phil,
Thank you for the update. According to your description, this issue has been partly resolved. Now it occurs with the “Public” site, but not the “Engineering” site. Am I right?
Regarding the “Public” site, I want to know if it is the SharePoint Online public facing website, or any team site?
If the public site is the SharePoint Online public facing website, you can possibly share me with the URL of the site and document, so that I can test the issue from my end.
Since this issue occurs with only one site, it may be caused by a corrupted web browser caching. I suggest you clear the cache and cookies of your Internet Explorer, and test this issue again.
Moreover, please also test this issue in any other web browser (For example: Firefox), and see if you can get the same result.
Best Regards,
Evan Zhang