ClaudeBot User-Agent & Blocking Rules
Operated by Anthropic. Verify the official user-agent string and safety status for this bot. Use the tools below to run a live crawlability test and get copy-paste robots.txt rules to manage its access. Crawler info last updated on .
What is ClaudeBot?
The main crawler for Anthropic. It scrapes the web to build the training dataset for the Claude family of AI models.
Test if ClaudeBot can access your URL
Enter URL to test if ClaudeBot is allowed or blocked.
Is ClaudeBot safe?
Yes, ClaudeBot is considered safe.
This bot is verified and generally respected by the SEO community.
For more information about our safety badges, visit our documentation.
User-Agent String used by ClaudeBot
ClaudeBotHow to block ClaudeBot with robots.txt?
Add the following standard robots.txt rule to prevent ClaudeBot from accessing your site.
User-agent: ClaudeBot Disallow: /
What happens if I block ClaudeBot?
Low Impact
Blocking ClaudeBot will only prevent AI training on your content. This will NOT affect your search engine rankings or website traffic. Safe to block if you want to opt-out of AI training.
Common use cases for ClaudeBot
Understanding how ClaudeBot is typically used can help you make informed decisions about whether to allow or block it on your website.
- Collects web content for AI model training
- Gathers data to improve language understanding
- Scrapes text, code, and other content for datasets
- May be used for chatbot and AI assistant development
- Typically does NOT affect search engine rankings