Claude-SearchBot User-Agent & Blocking Rules
Operated by Anthropic. Verify the official user-agent string and safety status for this bot. Use the tools below to run a live crawlability test and get copy-paste robots.txt rules to manage its access. Crawler info last updated on .
What is Claude-SearchBot?
Claude-SearchBot is a crawler used to index content specifically for search-related features within the Claude ecosystem.
Test if Claude-SearchBot can access your URL
Enter URL to test if Claude-SearchBot is allowed or blocked.
Is Claude-SearchBot safe?
Yes, Claude-SearchBot is considered safe.
This bot is verified and generally respected by the SEO community.
For more information about our safety badges, visit our documentation.
User-Agent String used by Claude-SearchBot
Claude-SearchBotHow to block Claude-SearchBot with robots.txt?
Add the following standard robots.txt rule to prevent Claude-SearchBot from accessing your site.
User-agent: Claude-SearchBot Disallow: /
What happens if I block Claude-SearchBot?
Low Impact
Blocking Claude-SearchBot will only prevent AI training on your content. This will NOT affect your search engine rankings or website traffic. Safe to block if you want to opt-out of AI training.
Common use cases for Claude-SearchBot
Understanding how Claude-SearchBot is typically used can help you make informed decisions about whether to allow or block it on your website.
- Collects web content for AI model training
- Gathers data to improve language understanding
- Scrapes text, code, and other content for datasets
- May be used for chatbot and AI assistant development
- Typically does NOT affect search engine rankings