PydanticAI MCP web search / 2026
How to Add Web Search to PydanticAI with MCP in 2026
PydanticAI’s MCP client is itself a toolset, so no adapter layer is needed between remote tool discovery and Agent. That makes a concise, typed search integration possible while leaving model selection and result validation in your Python application.
Quick answer
Attach an MCPToolset to a PydanticAI Agent.
Install PydanticAI with its MCP extra, construct MCPToolset with the hosted URL, and pass it through Agent(toolsets=[...]). PydanticAI treats an ordinary non-/sse URL as Streamable HTTP.
toolset = MCPToolset('https://webstractor.com/mcp')
agent = Agent(model, toolsets=[toolset])Before you begin
What you need
- Python 3.10 or newer
- A PydanticAI-supported model and provider credential
- The pydantic-ai MCP optional dependency
- An async application entry point
Step-by-step setup
Connect Webstractor to PydanticAI
Install the MCP client support
The full pydantic-ai package includes it; slim installations need the mcp optional group.
uv add "pydantic-ai-slim[mcp]"Create the remote toolset
Passing the HTTPS URL directly selects Streamable HTTP because the path does not end in /sse.
from pydantic_ai import Agent
from pydantic_ai.mcp import MCPToolset
webstractor = MCPToolset('https://webstractor.com/mcp')
agent = Agent(
'anthropic:claude-sonnet-4-6',
toolsets=[webstractor],
instructions=(
'Use search_web for public source discovery. '
'Cite source URLs and do not treat snippets as complete evidence.'
),
)Manage the lifecycle around related runs
PydanticAI can open and close the connection automatically. An async context is clearer when an application performs several calls together.
async with agent:
result = await agent.run(
'Find 5 public sources explaining Python free-threading. Include each URL.'
)
print(result.output)Add output validation for your product
If the application requires a fixed response, define a Pydantic output model containing a summary and source URLs. Validate final output separately from the MCP search feed so your UI contract stays under application control.
Tool selection
Use toolset preparation for narrower agents
A general Webstractor toolset exposes search, extraction, vertical search, and finance capabilities. For a search-only agent, use PydanticAI’s toolset preparation or filtering features to expose only the required actions.
This makes invalid tool choices less likely and lets tests assert the intended public-data boundary.
Error policy
Choose when the model may retry
PydanticAI’s MCPToolset supports tool_error_behavior. The default retry path is useful when the model can correct bad arguments, such as an excessive limit. Transport failures and restricted sources need bounded handling outside a model loop.
Record failed tool calls as diagnostics without placing secrets in logs.
Testing
Replace live tools in unit tests
Use deterministic test models or substitute toolsets for unit tests. Keep one integration test against the hosted server to verify initialization and tool discovery.
An evaluation should assert that source URLs in the final typed output came from the tool result, not from model memory.
Available data
What you can extract
- Discovered MCP tools attached as one toolset
- Bounded, source-linked public search results
- Markdown for model context
- Structured JSON suitable for Pydantic validation
AI workflows
Where normalized data helps
- Typed research agents
- Source-discovery steps before validated output
- Website-scoped public search
- Async Python assistants with observable tool calls
Boundaries
Public data only
- The remote server must be reachable when the agent uses the toolset.
- Search output remains public discovery data rather than verified truth.
- Tool errors should not trigger unbounded model retries.
webstractor.com does not bypass CAPTCHAs, login walls, paywalls, access controls, or regional restrictions. Review the source’s terms and applicable law before collecting or reusing data.
Common questions
PydanticAI and Webstractor FAQ
Do I need FastMCP in my application?
No. MCPToolset handles the client connection; Webstractor hosts the remote server.
Why use async with agent?
It explicitly manages registered toolset connections across one or more related runs.
Can I share one MCPToolset across agents?
Yes. PydanticAI documents entering the toolset context directly when sharing it across multiple agents.
Ready to try it?