How to Build a CrewAI Web Search Tool With the You.com Web Search API

TLDR: CrewAI agents get web search either through an MCP server attached via the mcps field, or through a custom tool that wraps an API call. The You.com Web Search API works with both: it is exposed as a remote MCP server at https://api.you.com/mcp, and it has a Python SDK for direct calls. This guide shows both wirings with working code, when to choose each, and the failure modes that make agent search quietly useless.

CrewAI agents run tasks, and many tasks need facts the model does not know: current prices, this week's news, a competitor's fresh docs. The framework gives you two ways to hand the agent a search capability, and picking between them is a real engineering decision, not a style preference.

What Are Your Options for a CrewAI Web Search Tool?

CrewAI's tool system covers built-in tools, community tools, and two extension paths documented in its official docs (2026-09-02): MCP servers attached to agents, and custom tools subclassing BaseTool. For web search specifically, the decision is between an attached MCP server and a custom wrapper around a search API.

The tradeoff to name out loud: MCP is the low-maintenance path, no tool code to write or keep compatible with framework versions, at the cost of controlling less between the agent and the API. A custom BaseTool is the control path: you own caching, validation, retries, and result shaping, at the cost of maintaining that wrapper yourself.

How Do You Attach the You.com MCP Server to a CrewAI Agent?

MCP is the open protocol for connecting AI clients to tools and data (modelcontextprotocol.io). CrewAI supports it natively through the mcps field on agents (CrewAI MCP docs, 2026-09-02). The You.com MCP server at https://api.you.com/mcp authenticates with a bearer API key or OAuth 2.1 and exposes every You.com API, including Web Search (You.com MCP docs, 2026-09-02).

The quick setup is a string reference in the mcps list, which is enough for the keyless profile:

mcps=["https://api.you.com/mcp?profile=free"]

The structured configuration is for when you need an Authorization header, tool filtering, or the tools-list cache:

import os
from crewai import Agent
from crewai.mcp import MCPServerHTTP

agent = Agent(
    role="Research Analyst",
    goal="Answer questions with current web sources",
    backstory="Analyst that always cites fresh sources.",
    mcps=[
        MCPServerHTTP(
            url="https://api.you.com/mcp",
            headers={
                "Authorization": f"Bearer {os.environ['YDC_API_KEY']}"
            },
            streamable=True,
            cache_tools_list=True,
        )
    ],
)
# The server's search tools are now available to the agent.

Those constructor options are documented in CrewAI's MCP integration guide (2026-09-02). For trying search without an API key, the keyless profile at https://api.you.com/mcp?profile=free works as the server URL with no header (You.com docs, 2026-09-02).

When Should You Write a Custom Tool Instead?

Three situations push toward a custom wrapper. You need a cache between the agent and the API, because agents re-ask similar questions and a cache cuts that cost. You need to validate or reshape results before the model sees them, for example stripping a result set down to URLs and snippets to protect the context window. Or you need retries and circuit breaking so a transient API error surfaces as a clean tool error instead of a crashed task.

CrewAI custom tools subclass BaseTool, set args_schema to a Pydantic input model, and implement _run (CrewAI tools docs, 2026-09-02). The input schema pattern comes from Pydantic, whose field documentation is at docs.pydantic.dev, and it is worth getting right: the field descriptions are what the agent reads when deciding how to call your tool. Here is a working web search tool wrapping the You.com Python SDK, with caching and real error handling:

import os
import time
from typing import Type
from pydantic import BaseModel, Field
from crewai.tools import BaseTool
from youdotcom import You

class WebSearchInput(BaseModel):
    query: str = Field(..., description="Search query, 3 to 6 words.")
    count: int = Field(5, description="How many results to return.")

class YouWebSearchTool(BaseTool):
    name: str = "web_search"
    description: str = (
        "Searches the live web and returns results with "
        "URLs, titles, and snippets. Use for anything "
        "requiring current information."
    )
    args_schema: Type[BaseModel] = WebSearchInput

    _cache: dict = {}
    _CACHE_TTL: int = 600

    def _run(self, query: str, count: int = 5) -> str:
        key = (query.lower().strip(), count)
        now = time.time()
        hit = self._cache.get(key)
        if hit and now - hit[0] < self._CACHE_TTL:
            return hit[1]

        try:
            with You(timeout_ms=30_000) as you:
                res = you.search(query=query, count=count)
        except Exception as exc:
            return f"SEARCH FAILED, do not answer from memory: {exc}"

        lines = []
        web = (res.results.web if res.results else []) or []
        for r in web[:count]:
            desc = (r.description or "")[:160]
            lines.append(f"{r.title}\n{r.url}\n{desc}")
        output = "\n\n".join(lines) or "NO RESULTS FOUND"

        self._cache[key] = (now, output)
        return output

Attach it to the agent with tools=[YouWebSearchTool()]. Three choices in that code are deliberate. The client is built with timeout_ms set, because the SDK otherwise inherits httpx's five-second default, which is tight for a live search (You.com Python SDK README, 2026-09-02). The error string tells the model not to answer from memory on failure, which is the difference between a degraded task and a hallucinated one. And results are stripped to title, URL, and a trimmed description, because raw result objects burn context the agent does not need. If you only need caching, CrewAI also caches tool results itself and exposes a cache_function hook on every tool for a custom policy (CrewAI tools docs, 2026-09-02); the wrapper above earns its keep when you also need reshaping and error handling.

How Do You Keep Agent Search Results Trustworthy?

The failure modes here are quiet ones, and each has a detection method.

The tool that is never called. The agent has the tool and still answers from training data, usually because the task description never says to verify. Detection: watch the tool call log for a task that should have searched. Zero calls on a current-events task means the prompt, not the tool, needs fixing.

The stale cache. Your wrapper caches aggressively and the agent confidently cites last week's price. Detection: set the TTL to the real freshness requirement, and log cache hits alongside tool calls so staleness is auditable after the fact.

The unfiltered firehose. Open search on a broad query returns forum noise, and the agent cites a Reddit thread as an authoritative source. Detection: this one shows up in output review. The fix is narrower queries (the Web Search API supports search operators and domain filters, including include_domains for up to 500 domains, per the You.com Search API docs, 2026-09-02), or pinning the tool to trusted domains in the wrapper itself.

The context blowout. Full-page extraction on every result fills the window before the agent reasons. Detection: token usage per task. The fix is the two-step pattern: search for snippets, then use the You.com Contents API (also on the same MCP server, up to 10 URLs per request) to pull the full page only for the result the agent actually needs.

Where Does This Fit With the Rest of the Stack?

The same server wires into other MCP clients. If your team also works in the terminal, our Claude Code web search tool guide covers the identical wiring from the CLI side. For the background on why structured retrieval beats scraping for feeding models, see the LLM web search API guide, and the company lookup guide shows the same primitives driving an entity-enrichment crew.

Next action: wire the MCP server to one agent with the keyless URL, give it a task that cannot be answered from memory, and watch the tool calls. If the agent searches, cites URLs, and completes the task, you have your baseline. Then decide whether your workload needs the custom wrapper, and if it does, start from the caching pattern above. API keys and usage rates are on the You.com platform and the pricing page.