Flowise URL extraction tutorial / 2026
How to Fetch and Extract Web Pages in Flowise with MCP in 2026
A Flowise agent often receives a URL rather than a search question. The extraction action is designed for that moment: it turns one known public page into readable Markdown or structured JSON while keeping the source URL available for citations.
Quick answer
Expose extract_public_url as a Flowise agent action.
Add Webstractor as a Custom MCP tool, enable extract_public_url, and instruct the Agent to use it only for a public URL already present in the user message or a previous search result. Markdown is the best default for summaries; JSON fits typed workflow branches.
{
"url": "https://en.wikipedia.org/wiki/Alan_Turing",
"format": "markdown",
"focus": "early computing work"
}Before you begin
What you need
- A Flowise Agent chatflow
- A tool-calling model configured in the Agent node
- The Webstractor Custom MCP connection
- A stable public URL for the first test
Step-by-step setup
Connect Webstractor to Flowise
Connect Webstractor in the Agent node
Add Custom MCP to the Agent tools and select Streamable HTTP. Use the hosted URL, save, and refresh Available Actions.
{
"type": "streamable-http",
"url": "https://webstractor.com/mcp"
}Enable only the page-reading action
Choose extract_public_url from Available Actions. For a dedicated URL summarizer, disabling unrelated search actions reduces ambiguity and makes the trace easier to evaluate.
Define URL and format rules
Require an explicit public URL and prefer Markdown when the next step is model reasoning. Use JSON when later Flowise nodes need fields such as type, title, or canonical URL.
instructions: |
When the user supplies a public URL, call extract_public_url once.
Use format=markdown for summaries and question answering.
Use focus only when the user names a topic within a long page.
Cite the returned canonical URL and never invent missing fields.Test a stable page and inspect the trace
Ask: “Read https://en.wikipedia.org/wiki/Alan_Turing and summarize the early computing work.” Confirm one extraction call, a non-empty Markdown result, and a final answer containing the source URL.
Routing
Use a deterministic URL gate before the agent
In a larger Flowise workflow, parse and validate the chat input before it reaches the Agent. Route messages containing a public URL to the extraction-capable branch and ordinary questions to the normal chat branch.
This prevents a model from manufacturing a URL merely to satisfy the tool schema and gives the user a clearer error when no URL was supplied.
Output choice
Markdown and JSON serve different nodes
Markdown preserves headings and readable content for summarization, question answering, and chunking. JSON is useful when a condition node branches on semantic type or when a downstream function stores selected metadata.
Do not ask for both formats in separate tool calls. Choose based on the immediate consumer; repeated representations of the same extraction share the canonical cached work.
Troubleshooting
Handle empty, restricted, and oversized sources
If the connection works but a specific URL fails, test another ordinary public HTML page. A source may be private, restricted, invalid, or too large. Return that limitation to the user instead of asking the agent to retry indefinitely.
If the model summarizes the URL without calling the tool, strengthen the instruction: content from a URL must come from extract_public_url, not model memory.
Available data
What you can extract
- Readable content from one public URL
- Canonical URL, title, author, and dates when available
- Optional focus for a topic within a generic page
- Semantic JSON entities for recognized public sources
AI workflows
Where normalized data helps
- Summarize a URL submitted to a Flowise chatbot
- Prepare source text for a classifier or evaluator node
- Read a selected result after a search step
- Create attributed context for support and research flows
Boundaries
Public data only
- The tool reads one public URL and does not crawl an entire site.
- Focus narrows generic pages but is not a selector language or completeness guarantee.
- Private, paywalled, login-only, or access-controlled content remains unavailable.
webstractor.com does not bypass CAPTCHAs, login walls, paywalls, access controls, or regional restrictions. Review the source’s terms and applicable law before collecting or reusing data.
Common questions
Flowise and Webstractor FAQ
Can Flowise fetch a PDF URL through this tool?
Yes, public remote PDF URLs can be extracted. Image-only PDFs without text are not OCRed.
What does focus do?
It asks extraction to emphasize a named topic on a generic page. It is optional and may not apply to semantic source adapters.
Should I put extracted Markdown directly in memory?
Usually keep the source record separate and pass only the relevant content into conversation memory to control context growth.
Ready to try it?