scrapeUrl

Extract text, HTML, or structured content from one or more web pages.

mindstudio.scrapeUrl({ url }) → Result

Extracts text, HTML, or structured content from one or more web pages, returning clean markdown by default — the workhorse for reading a page into a model. For a result set rather than a known URL, search first.

Parameters

url

string

RequiredURL(s) to scrape. Accepts a single URL, JSON array, or comma/newline-separated list

service

string

Scraping service to use

defaultfirecrawl

autoEnhance

boolean

No longer selects a provider — the default service's anti-bot engine decides per request how hard to work. Retained because existing workflows set it and the builder still renders it.

outputFormat

string

Output format: text returns markdown, html returns raw HTML, json returns structured scraper data, summary returns a model-written summary (Firecrawl only)

textjsonhtmlsummary

pageOptions

object

Page-level scraping options (content filtering, screenshots, headers, etc.)

Call from a method

import { mindstudio } from '@mindstudio-ai/agent';

const result = await mindstudio.scrapeUrl({
  url: /* ... */
});