Screaming Frog SEO Spider User-Agent & Blocking Rules
Operated by Screaming Frog
SEO Tools Caution
What is Screaming Frog SEO Spider?
The user-agent for the Screaming Frog SEO Spider desktop application. It is used by SEO professionals to audit websites.
Test if Screaming Frog SEO Spider can access your URL
Enter URL to test if Screaming Frog SEO Spider is allowed or blocked.
Is Screaming Frog SEO Spider Safe?
It is recommended to use caution with Screaming Frog SEO Spider.
This bot may have legitimate uses but can be resource-intensive or used for competitive intelligence.
For more information about our safety badges, visit our documentation.
User-Agent String used by Screaming Frog SEO Spider
Screaming Frog SEO SpiderHow to Block Screaming Frog SEO Spider?
Add the following standard robots.txt rule to prevent Screaming Frog SEO Spider from accessing your site.
User-agent: Screaming Frog SEO Spider Disallow: /