Allen Institute for AI Bots & Crawlers
The crawler directory currently lists 2 web crawlers operated by Allen Institute for AI.
About Allen Institute for AI
The Allen Institute for AI (AI2) operates AI2Bot for open-source AI research and dataset creation. Ai2Bot-Dolma is a specific crawler used to build the Dolma dataset, a massive open corpus for training language models. These crawlers support academic and open-source AI research initiatives.
2 Bots from Allen Institute for AI
| Bot Name | User-Agent | Category | Safety |
|---|---|---|---|
| AI2Bot | AI2Bot | AI Bots | Caution |
| Ai2Bot-Dolma | Ai2Bot-Dolma | AI Bots | Caution |