Agent Discovery Doctor — AI Crawler Index

Audit which agent-discovery documents a host actually serves — robots.txt, llms.txt, openapi.json, the .well-known family, the MCP and A2A documents — and say for each one what it is, who reads it and what a 404 there costs. It also validates a pasted llms.txt or a pasted A2A agent card against the spec, and drafts an llms.txt from a sitemap. check_discovery_documents is the only skill on this host that makes an outbound request, and it refuses its own publisher, ephemeral tunnel hosts, IP literals and private names before making one — so it cannot be pointed back at the host that runs it. Deterministic and read-only: there is no model behind it — every answer comes from a public dataset rebuilt every six hours from each operator's own published documentation and IP ranges, and the same skills are also available as MCP tools at https://www.pathwren.workers.dev/mcp/doctor. No key, no signup, no quota. Independent and unaffiliated with any operator it documents.

Skills

• Which discovery documents does this host serve?
  Probes 22 documents agents and trust indexes ask for — llms.txt, agent card, owners.json, oauth metadata, mcp.json, apis.json, openapi, robots, sitemap — as served, missing, gated or 200-with-HTML soft-404, and says who asks for each missing one. Refuses private, ephemeral and its own hosts. Example: host='example.com'.
  discoverywell-knownauditagents
• What is this document, and who reads it?
  One catalogue entry: what the document is for, the named clients observed asking this host for it with dates and the status they took, what a 404 costs, and the spec URL. No argument lists all 22. Example: name='owners.json' names the bot that asks for it twice.
  discoveryreferencedocumentationwell-known
• Check a pasted llms.txt
  Checks pasted llms.txt against the format: one H1, a blockquote summary, H2 sections of `- [name](url): notes`. Errors and warnings with line numbers and fixes, plus the parsed links. Text in, nothing fetched. Example: text='# Site' warns it has no summary and no sections.
  llms.txtvalidationspeclint
• Draft an llms.txt from a sitemap
  Paste sitemap.xml, or one URL per line, and get a draft llms.txt: URLs grouped into H2 sections by path, titles from slugs, lastmod kept, and a TODO wherever only you can write the sentence. A sitemap index is reported as one. Example: xml='https://e.com/docs/a\nhttps://e.com/blog/b'.
  llms.txtsitemapgeneratordraft
• Check a pasted A2A agent card
  Validates a pasted /.well-known/agent-card.json against the nine fields A2A marks required and each skill's id/name/description/tags, and warns on capabilities declared true that a reader will then try. Example: json='{"name":"a"}' returns the eight missing fields.
  a2aagent cardvalidationspec

How to call

https://www.pathwren.workers.dev/a2a/doctor

Listed in

Directories this entry was found in.