strale.io

https://api.strale.io

● healthy general

Fetch and parse a website's robots.txt. Returns rules per user-agent, sitemaps, crawl delays, and whether all crawlers are blocked. -- reliability 91.0/100 per Paygent Discover

How to call

cURL · x402-fetch / x402-axios will auto-handle 402

curl -X POST https://api.strale.io

/.well-known/x402.json

https://api.strale.io/.well-known/x402