FriendlyCrawler User-Agent & Blocking Rules
Operated by Unknown
Scrapers Caution
What is FriendlyCrawler?
A crawler that claims to be friendly, but often associated with marketing or SEO data collection.
Test if FriendlyCrawler can access your URL
Enter URL to test if FriendlyCrawler is allowed or blocked.
Is FriendlyCrawler Safe?
It is recommended to use caution with FriendlyCrawler.
This bot may have legitimate uses but can be resource-intensive or used for competitive intelligence.
For more information about our safety badges, visit our documentation.
User-Agent String used by FriendlyCrawler
FriendlyCrawlerHow to Block FriendlyCrawler?
Add the following standard robots.txt rule to prevent FriendlyCrawler from accessing your site.
User-agent: FriendlyCrawler Disallow: /