Ai2Bot-Dolma User-Agent & Blocking Rules
Operated by Allen Institute for AI
AI Bots Caution
What is Ai2Bot-Dolma?
A specific AI2 crawler used to build the Dolma dataset, a massive open corpus for training language models.
Test if Ai2Bot-Dolma can access your URL
Enter URL to test if Ai2Bot-Dolma is allowed or blocked.
Is Ai2Bot-Dolma Safe?
It is recommended to use caution with Ai2Bot-Dolma.
This bot may have legitimate uses but can be resource-intensive or used for competitive intelligence.
For more information about our safety badges, visit our documentation.
User-Agent String used by Ai2Bot-Dolma
Ai2Bot-DolmaHow to Block Ai2Bot-Dolma?
Add the following standard robots.txt rule to prevent Ai2Bot-Dolma from accessing your site.
User-agent: Ai2Bot-Dolma Disallow: /