Crawler IP Verifier — AI Crawler Index

Verify at volume that an address really belongs to the crawler operator it claims. Hand it up to 500 addresses and it says, for each, which operator prefix contains it, from which published source and when that source was last mirrored; hand it a CIDR and it reports every published prefix that contains, is contained by or overlaps it; ask it for an allowlist and it returns a paste-ready nginx, apache, haproxy, Cloudflare, ipset, caddy or plain-CIDR config. It also says which operators can be verified by prefix at all and which need the two-step reverse-DNS check — which it hands you as a command and never runs, because it makes no outbound request of any kind. Absence from a range list is reported as a miss, never as proof of a forgery. Deterministic and read-only: there is no model behind it — every answer comes from a public dataset rebuilt every six hours from each operator's own published documentation and IP ranges, and the same skills are also available as MCP tools at https://www.pathwren.workers.dev/mcp/netcheck. No key, no signup, no quota. Independent and unaffiliated with any operator it documents.

Skills

• Verify many addresses at once
  Check up to 500 addresses against every mirrored operator prefix list in one call. Returns, per address, the matching prefix, the source that published it and when it was mirrored, plus a rollup by operator. Accepts {ip, claim} pairs to test a claimed identity against the ranges, and says plainly that a miss is not evidence of a fake.
  ip rangesverificationbot detectionbatch
• Prefix arithmetic against published ranges
  Give a CIDR and get every published operator prefix that contains it, is contained by it, or partially overlaps it — the check for 'does my network collide with a crawler range' and for auditing an allowlist somebody handed you. Give an operator or source slug instead and get everything that source publishes.
  cidrip rangesnetworklookup
• Paste-ready allowlist or denylist
  Turn a set of operators into a config you can paste: nginx geo, nginx allow/deny, Apache, HAProxy, a Cloudflare firewall expression, an ipset script, a Caddy matcher, or a plain CIDR list. Every export carries a provenance header naming each source URL and the mirror time, and reports the rule cost.
  firewallnginxallowlistconfiguration
• Can this crawler be verified at all, and how?
  Per crawler: whether the operator publishes IP ranges, documents reverse DNS, or publishes no verification method at all — with the exact source URL or the exact dig commands, and how fresh this mirror is. Call it before reading anything into an address that matched nothing. Reverse DNS is never run here; the command is handed back instead.
  verificationreverse dnsoperatorsmethod
• The state of the mirror
  Per source: prefix counts, total addresses and /24-equivalents, smallest and largest prefix, the publishing URL and when it was last fetched; plus prefixes published by more than one operator. This is the sizing and freshness information an allowlist decision needs.
  freshnessmonitoringdata qualitymirror

How to call

https://www.pathwren.workers.dev/a2a/netcheck

Listed in

Directories this entry was found in.