Analyse robots.txt content and report whether a path may be crawled.
This server publishes no hosted endpoint. Run one of its packages locally.
scrape-le-mcp · npm 2.6.0 · stdio
claude mcp add --transport stdio scrape-le -- npx -y scrape-le-mcp@2.6.0