Developers (Framework) Bots & Crawlers
Our crawler directory currently tracks 1 web crawler operated by Developers (Framework). Analyze the safety statuses, bot categories, and user-agent strings for every bot in this operator's fleet. Operator info last updated on .
About Developers (Framework)
Scrapy is a popular open-source web crawling framework. Seeing this user-agent usually means a developer hasn't set a custom name for their bot. The actual operator varies by implementation.
1 Bot from Developers (Framework)
| Bot Name | User-Agent | Category | Safety |
|---|---|---|---|
| Scrapy | Scrapy | Scrapers | Aggressive |