Back to code-only MCP servers

site-crawler-mcp

updated this year

Site-wide crawler MCP server: BFS crawl of all internal links with time/title/URL filters, page operations (summarize/click/download), SQLite persistence. Built on crawl4ai.

Code-only MCP server: no hosted endpoint, so you install and run it yourself. It is not probed, graded or counted in our reachability statistics.

Get it

Repository
github.com/coolaigit/site-crawler-mcp
License
MIT
Last update
2026-08-18
GitHub stars
★ 0

Listed in

Run this server as a hosted endpoint? Submit the endpoint and it moves to the remote directory, where it is probed and graded.