OpenHands URL extraction / 2026
How to Fetch Web Pages in OpenHands with Webstractor MCP in 2026
A coding agent often has the exact documentation, issue, product, or article URL but still needs a compact representation it can reason over. Webstractor gives OpenHands a read-only URL tool that removes page-interface noise while preserving the source link. This is a different workflow from web search: use extraction when the URL is already known.
Quick answer
Give OpenHands a clean reader for known public URLs.
Connect the hosted Webstractor endpoint through OpenHands HTTP transport, then ask OpenHands to call extract_public_url with a specific URL. Use Markdown for reading and JSON only when the task needs typed fields.
openhands mcp add webstractor --transport http https://webstractor.com/mcpBefore you begin
What you need
- OpenHands CLI 1.0 or later
- A public HTTP or HTTPS page to read
- Tool calling enabled for the selected model
Step-by-step setup
Connect Webstractor to OpenHands
Add the hosted reader
Register the endpoint with the HTTP transport. The command writes the persistent user-level MCP configuration.
openhands mcp add webstractor --transport http https://webstractor.com/mcpStart a fresh conversation
Run OpenHands, enter /mcp, and confirm that webstractor is active. Configuration changes made during an existing conversation appear as pending until the next conversation.
openhands
# In the conversation, enter:
/mcpValidate extraction with a focused request
Provide the exact public URL, request Markdown, and name the topic. The focus hint helps a long generic page return the section that matters without turning the request into a site crawl.
# Paste into OpenHands
Use webstractor extract_public_url to read https://docs.astro.build/en/guides/upgrade-to/v6/ as Markdown with focus "breaking changes". List only changes relevant to this repository and cite the URL.Tool choice
Use extraction only after you have a URL
If a task starts with a named URL, call extract_public_url directly. If it starts with a question and no known source, search first and extract only a promising result. This distinction avoids unnecessary discovery requests.
Markdown is best for comprehension and summarization. JSON is useful when an OpenHands task needs stable fields such as semantic type, title, author, dates, or collection items.
- Known URL → extract_public_url
- Unknown source → search_web, then extract
- Human-readable evidence → Markdown
Troubleshooting
Separate connection errors from page errors
If /mcp does not list webstractor, inspect `openhands mcp get webstractor` and restart the conversation. If only one URL fails, test another ordinary public page: the original may require login, block automated access, or be unavailable.
A focused request can legitimately omit unrelated sections. Retry without focus when completeness matters, then narrow the result in the prompt.
Available data
What you can extract
- Readable Markdown from a known public URL
- Optional focused extraction for a topic such as migration or pricing
- Normalized JSON for typed application logic
- Canonical URL and public metadata for attribution
AI workflows
Where normalized data helps
- Read a migration guide before updating dependencies
- Summarize a linked issue or public article
- Compare documentation statements with repository code
- Extract one relevant section from a long reference page
Boundaries
Public data only
- The tool reads one public URL; it does not crawl an entire site.
- Login walls, paywalls, and access controls are not bypassed.
- Do not send URLs containing secrets, session tokens, or private hostnames.
webstractor.com does not bypass CAPTCHAs, login walls, paywalls, access controls, or regional restrictions. Review the source’s terms and applicable law before collecting or reusing data.
Common questions
OpenHands and Webstractor FAQ
Can OpenHands read a URL without searching for it?
Yes. Pass the exact public URL to extract_public_url; no search step is needed.
Should I request Markdown or JSON?
Choose Markdown when the model needs to read the page. Choose JSON when downstream logic needs normalized fields.
Does this let OpenHands access private documentation?
No. Webstractor accepts public URLs and does not take site credentials or bypass access controls.
Ready to try it?