news-please User-Agent & Blocking Rules
Operated by Open Source
Scrapers Unsafe
What is news-please?
An open-source news crawler and extractor library.
Test if news-please can access your URL
Enter URL to test if news-please is allowed or blocked.
Is news-please Safe?
No, news-please is generally considered unsafe.
This bot typically consumes server resources without providing search ranking value.
For more information about our safety badges, visit our documentation.
User-Agent String used by news-please
news-pleaseHow to Block news-please?
Add the following standard robots.txt rule to prevent news-please from accessing your site.
User-agent: news-please Disallow: /