anansi

active

A self-healing scraper for hostile sites: broken selectors repair themselves, browser rendering kicks in when needed, and a coherent identity layer (Chrome TLS fingerprints, matched personas, vendor-aware Cloudflare/Akamai/DataDome handling) works to slip past bot detection. Ships with an MCP server so any LLM can drive a full crawl by conversation.

Code-only MCP server: no hosted endpoint, so you install and run it yourself. It is not probed, graded or counted in our reachability statistics.

Get it

Repository: github.com/mdowis/anansi
License: Apache-2.0
Last update: 2026-09-02
GitHub stars: ★ 117

adaptive-scraping ai-agent anti-bot concurrent-processing crawler data-extraction llm-tools mcp mcp-server pydantic python self-healing

Listed in

github · source

Run this server as a hosted endpoint? Submit the endpoint and it moves to the remote directory, where it is probed and graded.