SemrushBot User-Agent & Blocking Rules
Operated by Semrush
SEO Tools Caution
What is SemrushBot?
SemrushBot is the main crawler for Semrush, an all-in-one marketing toolkit. It collects data for SEO analysis and site audits.
Test if SemrushBot can access your URL
Enter URL to test if SemrushBot is allowed or blocked.
Is SemrushBot Safe?
It is recommended to use caution with SemrushBot.
This bot may have legitimate uses but can be resource-intensive or used for competitive intelligence.
For more information about our safety badges, visit our documentation.
User-Agent String used by SemrushBot
SemrushBotHow to Block SemrushBot?
Add the following standard robots.txt rule to prevent SemrushBot from accessing your site.
User-agent: SemrushBot Disallow: /