Claude-Web User-Agent & Blocking Rules
Operated by Anthropic. Verify the official user-agent string and safety status for this bot. Use the tools below to run a live crawlability test and get copy-paste robots.txt rules to manage its access. Crawler info last updated on .
What is Claude-Web?
Claude-Web is used when a Claude AI user asks the model to visit a specific URL to answer a question.
Test if Claude-Web can access your URL
Enter URL to test if Claude-Web is allowed or blocked.
Is Claude-Web safe?
Yes, Claude-Web is considered safe.
This bot is verified and generally respected by the SEO community.
For more information about our safety badges, visit our documentation.
User-Agent String used by Claude-Web
Claude-WebHow to block Claude-Web with robots.txt?
Add the following standard robots.txt rule to prevent Claude-Web from accessing your site.
User-agent: Claude-Web Disallow: /
What happens if I block Claude-Web?
Low Impact
Blocking Claude-Web will only prevent AI training on your content. This will NOT affect your search engine rankings or website traffic. Safe to block if you want to opt-out of AI training.
Common use cases for Claude-Web
Understanding how Claude-Web is typically used can help you make informed decisions about whether to allow or block it on your website.
- Collects web content for AI model training
- Gathers data to improve language understanding
- Scrapes text, code, and other content for datasets
- May be used for chatbot and AI assistant development
- Typically does NOT affect search engine rankings