ai-crawler-policy

Apify MCP Server

apify15.2K15,228 GitHub stars

Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡

Firecrawl MCP Server

firecrawl7.8K7,777 GitHub stars

Firecrawl, the context API for AI agents: search, scrape, interact with the web, plus paper search.

exa

exa5.2K5,216 GitHub stars

Connect AI agents to Exa for web search, content fetching, and multi-step research.

Brave Search MCP Server

brave1.6K1,565 GitHub stars

Brave Search MCP Server: web results, images, videos, rich results, AI summaries, and more.

Real-time web search, reasoning, and research through Perplexity's API

Real-time web search, reasoning, and research through Perplexity's API