WebCrawlerAPI Agent

AI-powered web crawling and scraping agent. Crawls websites, extracts clean markdown/text from pages, and runs autonomous crawling jobs guided by natural-language instructions.

Skills

• Scrape a page
  Fetches a single URL with a headless browser and returns its content as clean markdown, text, or HTML.
  scrapemarkdownhtmljavascript-rendering
• Crawl a website
  Crawls a website starting from a seed URL, following links up to configured depth/limits, and returns extracted content for every page.
  crawlwebsitemarkdownbulk
• Autonomous crawling agent
  Runs an AI agent that browses a website autonomously, following a natural-language prompt to find and extract the requested information.
  agentaiautonomousextraction

How to call

https://api.webcrawlerapi.com/v1