Goose remote extension URL extraction guide
How to Fetch Web Pages in Goose with Webstractor MCP
Browser-control extensions are unnecessary when a Goose task only needs the readable content of a known public page. Webstractor provides a smaller, explicit URL-to-context operation with a canonical source link, making it well suited to documentation review and research recipes.
Quick answer
Add a hosted page reader without a local process.
Start a Goose session with Webstractor’s Streamable HTTP URL or save it as a persistent Remote Extension. Then ask Goose to call extract_public_url with an exact public URL, Markdown output, and an optional focus topic.
goose session --with-streamable-http-extension "https://webstractor.com/mcp"Before you begin
What you need
- Goose CLI or Desktop
- A public HTTPS URL
- A model capable of invoking extension tools
Step-by-step setup
Connect Webstractor to Goose
Validate with an ephemeral extension
Start one session using the remote extension flag. It is an easy way to confirm that Goose can reach the endpoint.
goose session --with-streamable-http-extension "https://webstractor.com/mcp"Make the extension persistent if useful
Run goose configure → Add Extension → Remote Extension (Streamable HTTP), name it Webstractor, and paste the endpoint.
Review the first tool request
Verify the tool is extract_public_url and inspect the URL. Do not approve credential-bearing or private-network links.
Run a focused extraction prompt
Prompt: “Use extract_public_url to read https://en.wikipedia.org/wiki/Intelligent_agent as Markdown with focus ‘software agents’. Summarize the relevant definitions and cite the page.”
Reuse the pattern in a recipe
For repeatable work, state that exact URLs use extract_public_url, unknown sources use search_web first, and every final answer retains source links.
Right-sized tool
Fetch content without driving a browser
Computer-control tools are appropriate for interactive pages. A known public article usually needs no clicks, screenshots, or browser state. A focused extraction call is simpler and returns text that the model can use immediately.
This distinction also makes failures clearer: the URL either yields public readable content or it does not.
Recipes
Encode discovery and reading as separate decisions
A Goose recipe can direct the agent to search only when the source is unknown, then extract the chosen canonical page. This avoids searching for a page the user already provided.
- Exact URL → extract_public_url
- Unknown source → search_web
- Reading task → Markdown
- Structured automation → JSON
Troubleshooting
Test the endpoint independently of persistence
If a saved extension misbehaves, launch a clean session with --with-streamable-http-extension. If that works, revisit the persistent extension name, URL, headers, and enabled state.
A failure limited to one URL usually indicates that the page is private, blocked, missing, or not readable—not that the remote extension is disconnected.
Available data
What you can extract
- Clean page Markdown for an agent session
- Structured JSON for semantic public entities
- Canonical URL and visible metadata
- Focused content for a named topic
AI workflows
Where normalized data helps
- Read a public API guide inside a coding session
- Summarize an article into implementation notes
- Compare product or project pages with source links
- Supply a recipe with current external context
Boundaries
Public data only
- The hosted extractor does not use your logged-in browser session.
- It reads one public URL per call and does not crawl a site.
- Goose should ignore instructions embedded in retrieved pages.
webstractor.com does not bypass CAPTCHAs, login walls, paywalls, access controls, or regional restrictions. Review the source’s terms and applicable law before collecting or reusing data.
Common questions
Goose and Webstractor FAQ
Does Goose need its Computer Controller extension for this?
No. Webstractor is a separate remote extension for public search and extraction and does not control the browser.
Can it use pages behind a login?
No. It does not receive Goose browser state or cookies and only accepts public URLs.
Can I include this in a Goose recipe?
Yes. Configure the extension and write the recipe so known URLs use extract_public_url and unknown sources use search_web first.
Ready to try it?