Our check bot

The user agent

It never sends Googlebot’s, GPTBot’s or any other crawler’s user agent. Testing a site with those is part of the paid audit, and only with the site owner’s permission.

What it fetches

Only when a person asks for a check of your site, and at most 14 requests per check:

• your homepage, following up to 3 redirects itself (each redirect is one request);
• the plain http:// address of your homepage, to see whether it sends visitors to the secure one;
• /robots.txt, once for each site address the redirects lead to;
• the sitemap your robots.txt names, or /sitemap.xml, on your own domain only, and only the first sitemap file;
• /llms.txt;
• the address your homepage’s canonical tag names.

What it reads from those pages is cached for 24 hours, and we limit how often one site can be checked, so a popular site is not fetched over and over.

What it never does

• It never sends cookies or logs in, never submits forms, and never runs JavaScript or follows a meta refresh.
• It stops reading a page past a size limit (2 MB for a page) and gives up on slow responses.
• It reads public pages only and keeps the values it checks, not copies of your pages.

How to block it

Add this to your robots.txt. If robots.txt disallows ProudSiteCheck, the checker stops and tells the person who asked that your site does not allow it.

User-agent: ProudSiteCheck
Disallow: /

The speed part of the check asks Google’s PageSpeed Insights to test your homepage. Google’s own systems make that request, not ours, so this rule does not cover it.

Questions or problems

If the checker is causing you any trouble, email hello@proudsite.co.uk and we will look into it.