MCP servers that ship as code — a repository or a package you install and run locally (usually over stdio). They have no hosted endpoint, so we cannot probe or grade them: instead each one shows its repository, install command, license, stars and when it was last updated.
Collected from GitHub repositories and npm, PyPI, Cargo, Go, Maven, NuGet and RubyGems packages tagged as MCP servers, the official MCP Registry, Docker's MCP catalog, ModelScope, Baidu MCP World, Cline and other public directories; one entry per repository. A server that also offers a hosted endpoint is listed under Remote instead. These entries are not counted in our reachability, verification or readiness statistics.
-
serpstat-crawlingstale
Serpstat SERP Crawling API MCP Server
npm MIT -
—
MIT ★ 5 -
自定义项目名,获取相关文章,建立相关数据库
-
MCP server for crawling web pages and converting to markdown
npm MIT -
Medical Data Crawler MCP Server
★ 1 -
crawlgithub.com/vibheksoni/crawlupdated this year
Async web search, fetch, crawl, and screenshot tooling with SDK, CLI, and MCP interfaces.
★ 3 -
enhanced-crawl-mcpgithub.com/milhy545/enhanced-crawl-mcpupdated this year
Production-ready FastAPI microservice for web crawling optimized for AI/LLM consumption
★ 1 -
crawl-mcpgithub.com/gqy20/crawl-mcpupdated this year
基于 crawl4ai 和 FastMCP 的 MCP 服务器,提供网页爬取和 AI 分析功能
MIT ★ 1 -
@groundcraft/crawlgithub.com/humphreytheodore/groundcraftupdated this year
Groundcraft Crawl — scrape, search, map and crawl the web through one metered API. CLI, SDK and MCP server.
npm MIT -
反爬与浏览器自动化工具包:审计、体检、冒烟、MCP
MIT -
MCP server for fetching, crawling, and analyzing websites. 8 tools: fetch, crawl, extract, SEO audit, sitemap, broken links, screenshots. Anti-bot protections, throttling, robots.txt compliance.
-
微信公众号文章抓取 MCP 服务器 - 支持自动图片下载、内容清理、智能抓取,可生成完整的本地化Markdown文档
npm MIT -
crawl-mcpupdated this year
MCP server to fetch raw HTML content from Chrome pages via Chrome DevTools Protocol
npm MIT -
mcp-crawl-paritygithub.com/arturseo-geo/mcp-crawl-parityupdated this year
MCP server for Googlebot vs AI crawler parity analysis — compare crawl behaviour from Nginx logs with GSC data to detect visibility gaps
MIT ★ 1 -
Crawl Engine - embeddable TypeScript web crawler with an MCP server for SEO, link, and schema analysis.
MIT ★ 1 -
Crawl Reddit posts + comments over Atom RSS when .json/api are blocked. REST + MCP service, curated markdown for knowledge bases.
MIT -
mcp-common-crawlgithub.com/arturseo-geo/mcp-common-crawlupdated this year
MCP server for Common Crawl CDX — backlink discovery across 3 indexes, expired/parked domain finder, competitor gap analysis. Free alternative to $100/mo backlink APIs. By TheGEOLab.net
NOASSERTION ★ 1 -
Naver Finance web crawler with MCP server support - crawl Korean stock market data
npm MIT -
AI-powered web crawler — self-hosted Firecrawl alternative. Crawl, extract, render JS, search. CLI + MCP + REST API + Dashboard.
npm Apache-2.0 -
claude-browser-crawl-cdpgithub.com/gruncode/claude-browser-crawl-cdpupdated this year
Drive a real, logged-in Chrome from Claude Code over CDP — a persistent Playwright REPL, ~25x faster web crawling & automation.
MIT -
hotelrate-crawlgithub.com/seanbabalala/hotelrate-crawlupdated this year
Real-time hotel price comparison across Trip.com, Booking.com, Agoda & Google Hotels — MCP server for AI agents (Claude, Cursor, etc.)
MIT ★ 12 -
Run the fastCRW web crawler natively on Windows: tray on/off switch, JS rendering via Chrome, MCP for AI agents, optional sharing over Tailscale.
MIT -
Keep existing Firecrawl CLI/Skill calls unchanged, fall back to DeepSeek when credits run out, or use the built-in Agent CLI, Skill, and MCP directly.
MIT ★ 1 -
The agent-readiness scanner and fixer for WordPress — run the 21-check readiness scan (Level 0-5) with evidence, fix failures in one click, and publish Markdown negotiation, llms.txt, MCP server card, agent skills, AI bot rules, and more.
★ 1 -
DJAI Academy community-built, free open-source technical SEO crawler and Screaming Frog alternative with JavaScript audits, reports, a local dashboard, and an MCP server for AI agents. Theoretical 100M+ URL architecture.
MIT ★ 7 -
MCP server that checks whether a website is set up for AI the way its owner intends: can GPTBot, ClaudeBot, PerplexityBot and 50+ other AI crawlers read it, and what does its access policy allow? Generates llms.txt, an AI-crawler robots.txt and JSON-LD, reads your Answer Analysis, and checks whether a page permits an AI use.
npm MIT -
anycrawl-mcpupdated this year
AnyCrawl MCP Server - Adds powerful web scraping and crawling to Cursor, Claude and any other LLM clients
npm MIT -
anycrawl-mcp-serverupdated this year
AnyCrawl MCP Server - Adds powerful web scraping and crawling to Cursor, Claude and any other LLM clients
npm MIT -
@iflow-mcp/any4ai-anycrawl-mcp-serverupdated this year
AnyCrawl MCP Server - Adds powerful web scraping and crawling to Cursor, Claude and any other LLM clients
npm MIT -
webmesh-mcpgithub.com/creatorpiyush/webmesh-mcpupdated this year
Token-economical web browsing, scraping, and crawling MCP server. Tiered fetching (static → browser), robots.txt compliance, session persistence, and BFS crawling.
npm MIT -
mcp-web-agentgithub.com/creatorpiyush/mcp-web-agentupdated this year
Token-economical web browsing, scraping, and crawling MCP server. Tiered fetching (static → browser), robots.txt compliance, session persistence, and BFS crawling.
npm MIT -
mcp-crawl4aigithub.com/wyattowalsh/crawl4ai-mcpupdated this year
Model Context Protocol server for web crawling powered by Crawl4AI
pypi MIT -
exa-mcpgithub.com/2aronS/exa-mcpupdated this year
MCP server for web search and crawling using Exa API
MIT ★ 6 -
@octivas/mcpgithub.com/octivas/octivas-mcpupdated this year
MCP server for Octivas web scraping, crawling, and search API
npm MIT -
Tiny round-robin crawl MCP backed by Firecrawl, Tavily, and Scrape.do
npm MIT -
Exa MCP for web search and web crawling!
npm MIT ★ 5,235 -
A Model Context Protocol (MCP) server that provides search and crawl functionality using Search1API
npm MIT -
@beejay141/docs-mcpupdated this year
MCP server that crawls documentation websites and exposes structured tools for AI agents to search and read library docs
npm ISC -
webcrawl-mcpgithub.com/andyliszewski/webcrawlupdated this year
Lightweight MCP server for web scraping, search, and crawling — local-first with optional Firecrawl fallback
pypi MIT -
@vermin/mcpactive
Vermin MCP server (stdio): scrape, crawl, map and brand tools for Claude, Cursor, VS Code and other MCP clients.
npm MIT -
The web, saved for your agents. Local-first web retrieval: fetch, extract, save, search, crawl and cite via CLI and MCP.
npm Apache-2.0 -
A Model Context Protocol (MCP) server providing ethical web crawling and search capabilities
npm MIT -
webcontext-aigithub.com/sumeeth-24/webscrapper-aiupdated this year
Turn any web content into clean AI-ready context — with crawling, chunking, semantic search, and MCP tools
npm MIT -
Model Context Protocol server for Spider Cloud web scraping and crawling API - LinkedIn profiles, anti-bot bypass, and more
npm MIT -
searchclaw-mcpupdated this year
MCP server for SearchClaw — Search, Extract, Crawl in One API. The complete web data pipeline for AI agents.
npm MIT -
sarvcrawl-mcpactive
MCP server for SarvCrawl — 21 tools to scrape, crawl, search and manage knowledge bases from Claude Code, Cursor, VS Code, and any MCP-compatible AI client
npm Apache-2.0 -
anti-botgithub.com/TaroYamada08/anti-botupdated this year
Its fetchers bypass anti-bot systems like Cloudflare Turnstile out of the box. And its spider framework lets you scale up to concurrent, multi-session crawls. 🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
BSD-3-Clause ★ 1 -
Local MCP for web search, direct-URL fetch, site crawling, docs export, and bounded smartbrowser local browser actions.
npm SEE LICENSE IN LICENSE -
claude-code-mcpsgithub.com/arturseo-geo/claude-code-mcpsupdated this year
5 MCP servers for Claude Code — SEO page auditor, SERP intelligence, Common Crawl backlinks, advanced GSC, WordPress management. By TheGEOLab.net
MIT ★ 2 -
seoh-mcpupdated this year
MCP server for SEOh! — GEO scoring, SEO audits, competitor comparison, keyword analysis, technical checks, schema validation, SERP preview, site crawling, full GEO reports, and usage tracking
npm MIT -
The focused-crawling MCP server for AI agents. Rank a whole site by your topic, read any page as clean Markdown, extract anything
pypi MIT ★ 12 -
Anakin plugin for OpenCode: Anakin's hosted MCP server (22 tools for web scraping, search, crawling, deep research, Wire site actions, monitors and browser tasks) plus Anakin skills
npm MIT -
quanticdata-mcpactive
MCP server for the QuanticData API — scrape, search, map, crawl the web and pull residential/datacenter/mobile proxies for AI agents. Connect QuanticData to Claude, Cursor, and any MCP client.
npm MIT -
SpiderHead — local-first SEO toolkit in Rust: CLI and MCP server for AI agents. Server log analysis with verified search/AI bots, crawling, audits, projects. Desktop app is early.
MIT -
quantumproxies-mcpactive
MCP server for the QuantumProxies API — scrape, search, map, and crawl the web through residential proxies, and hand AI agents ready-to-use proxies of every type (residential, mobile, datacenter, ISP, IPv6). Connect QuantumProxies to Claude, Cursor, and a
npm MIT -
A web crawling and conversion system that uses FastAPI and MCP to crawl websites and convert them into markdown.
MIT -
A powerful Model Context Protocol (MCP) server for crawling websites and extracting assets including images and SEO metadata. Built for e-commerce sites and general web crawling needs.
MIT ★ 6 -
MeshArc web crawler for Claude Code: search, scrape, map and crawl sites, and see what changed between crawls.
MIT -
crawl4ai-mcp-servergithub.com/sadiuysal/crawl4ai-mcp-serverupdated this year
🕷️ A lightweight Model Context Protocol (MCP) server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents. Similar to Firecrawl's API but self-hosted and free. Perfect for integrating web scraping into your AI workflows with OpenAI Agents SDK, Cursor, Claude Code, and other MCP-compatible tools.
pypi MIT ★ 118 -
MCP server and CLI that crawls websites for AI agents: finds the answer page 2x more often than Crawl4AI, 10x cheaper per answer than letting Claude crawl.
Apache-2.0 ★ 89