AI Crawler Checker — Robots.txt Policy for Six Bots
- by kriton-dev
- Hosted
- v1.0.2
Check declared robots.txt permissions for six AI crawlers at URL paths
claude mcp add --transport http 'ai-crawler-access-check' 'https://mcp.apify.com/?tools=hermeskd/ai-crawler-access-check'Run in a terminal; adds the server to Claude Code.
Connect, 1 way to connect
- Remote
https://mcp.apify.com/?tools=hermeskd/ai-crawler-access-checkAuthorization
Activity
- Published
