URL Metadata API

Free URL metadata extraction (rate-limited)

Full metadata extraction

Text-only summary

Title + description + keywords

Open Graph + Twitter Card tags

robots.txt policy summary

DKIM (RFC 6376) selector record lookup

llms.txt (Answer.AI) standard parser

AI bot policy audit (22 crawlers)

DNS record lookup (A/AAAA/MX/NS/TXT/CNAME) via Cloudflare DoH

WHOIS / RDAP lookup via rdap.org

HTTP security header audit (CSP/HSTS/X-Frame-Options/etc.)

HTTP redirect chain + final URL

TLS certificate audit (issuer, expiry, signature alg, SAN list)

Page load performance (TTFB, size, compression, redirects)

Tech-stack fingerprint (frameworks, libraries, server, CDN, fonts, analytics)

Page carbon footprint (bytes, gzip ratio, CO2 per visit, A-F grade)

RSS/Atom feed discovery + parse (first 10 entries)

sitemap.xml discovery + URL extraction (nested sitemaps, .gz support)

JSON-LD structured data extraction (schema.org types, @graph support)

Hyperlink extraction (internal/external classification, nofollow, top external domains)

HTML form extraction (action, method, inputs, form-type guess)

Email + contact + social extraction (mailto, obfuscated [at]/[dot])

Readability metrics (Flesch Reading Ease, Flesch-Kincaid grade, Gunning Fog, reading time, A-F grade)

Script/iframe inventory (first/third-party classification, SRI, inline sizes, language hints)

Meta-refresh + JS-redirect detector (canonical mismatch, SEO risk)

hreflang + x-default multilingual SEO audit

Microdata + RDFa structured data extraction (schema.org types, itemprop, RDFa property/vocab)

Content-Security-Policy parser (directives, sources, nonce/hash, unsafe-* detection)

Permissions-Policy header audit (browser feature gating, high-risk feature detection)

GDPR/CCPA cookie consent audit (CMP detection, cookie categorization, security attrs)

HTML heading-hierarchy audit (h1-h6 counts, sequence validation, multiple-h1, missing-h1, A-F grade)

Set-Cookie security-attribute audit (Secure/HttpOnly/SameSite/Max-Age/Size/Priority/Partitioned + A-F grade)

Comprehensive SEO audit (title length, meta description, canonical, robots, viewport, OG/Twitter, JSON-LD, hreflang, favicon, sitemap reference + weighted A-F grade)

WCAG-light accessibility audit (html lang, alt-text coverage, label/aria-label on inputs, ARIA roles, table th scope, button accessible name, tabindex, autocomplete + A-F grade

Comprehensive favicon + app-icon extractor (link rel=icon/apple-touch-icon/mask-icon, manifest.json icons, msapplication-TileImage, og:image, Twitter image)

Full llms.txt + llms-full.txt fetcher with parsed section breakdown (H1 title, per-section URL + bullet count) for AI-agent context loading

Privacy-signal detection audit (DNT/GPC/AI-opt-out meta tags, navigator.doNotTrack + globalPrivacyControl, Sec-GPC detection, privacy-policy link check, A-F grade)

Third-party embed/iframe/object inventory (YouTube/Vimeo/Wistia/GoogleMaps/Stripe/Calendly/Typeform/Spotify/Soundcloud/Twitch/Figma/Canva/Loom/Twitter/IG/CodePen/JSFiddle/CodeSandbox/Replit/SlideShare/SpeakerDeck/Issuu/Miro/Airtable, sandbox/allow/loading/referrerpolicy capture, A-F grade)

<img>/<picture>/<source>/<svg>/<canvas>/<video> inventory with first/third-party classification, lazy-loading adoption, modern format hints, missing-alt audit, A-F grade

Top anchor-text frequencies + internal/external classification + nofollow/sponsored/ugc counts + empty-anchor + generic anchor-text detection, A-F grade

TLS handshake + certificate audit (TLS version, cipher, ALPN, cert chain, days-to-expire, key type/size, SAN coverage, sig algorithm, A-F grade)

Outbound link/redirect health audit (dead-link probe, redirect chain depth, mixed-content detection, broken in-page anchors, invalid mailto, A-F grade)

Multi-signal page-type classifier (URL/title/JSON-LD/OG/microdata/forms/structure -> primary category + confidence + secondary + A-F grade)

Composite email-auth deliverability score (SPF + DKIM + DMARC + MTA-STS + BIMI rollup, A-F grade, post-Feb-2024 Gmail/Yahoo bulk-sender compliance check)

Content-freshness detector (publish/modified dates from meta/JSON-LD/HTTP headers/time elements, age in days, A-F grade)

External-resource inventory (script/stylesheet/font/image/iframe hosts, top domains, 22-family ad/tracker classifier, third-party dependency A-F grade)

Composite agent-trustworthiness score — security headers + discoverability + structured data + cookie consent rollup with A-F grade and prioritized findings

Page render-mode + hydration-profile detector — framework hints (Next/Nuxt/Remix/Astro/SvelteKit/React/Vue/Angular) + body content size + hidden DOM % + noscript fallback + verdict for AI-agent fetching strategy

Unified structured-data audit — JSON-LD + Microdata + RDFa aggregated into unique schema.org types + parent-class hierarchy + 21 rich-result candidate detection + machine_readability_score 0-100, A-F grade