Search and Contents: Integration Runbook for Inference Providers

1. Start here

Choose one path:

Use this sequence:

1. Obtain a provider API key with Search and Contents access.
2. Select the model-controlled fields and provider defaults in section 3.
3. Implement path A in section 4 or path B in section 5.
4. Add the runtime controls in section 6.
5. Record your integration decisions in section 7.
6. Pass the acceptance tests in section 8 before production.

Recommended flow: Search with highlights. Select and deduplicate sources. Read selected URLs with Contents. Generate an answer with source links.

This separation matters when the model proposes parallel queries. Full-page extraction retrieves content before the host selects useful pages. One measured full-page response was 1.7 MB. This is an observed example, not a typical size or an API limit. Several such responses can exceed message and context limits.

Keep defaults in code. Do not depend on the model to repeat them in each call. Use normalization for query syntax. Use host controls for permissions, budgets, and retries.

2. Terms and ownership

The provider must authenticate its customers. Keep the You.com key on the server. Do not put it in model context, browser code, tool results, or logs.

Apply customer quotas before calls use the shared key. Keep customer-specific overrides and cached results separate. Do not mutate shared request headers for concurrent customers. Share cached results only when your isolation and access policy permits it.

3. Parameter control and defaults

3.1 Select the model-controlled fields

Start with query and optional freshness. Expose more fields only when your model and workload tests support them.

site:, lang:, and loc: can affect fields even when those fields are absent from the model schema. Section 4.7 defines this behavior. If policy must forbid these filters, enforce that policy in host code.

3.2 Resolve values without losing defaults

First validate each input source. Reject invalid model arguments and invalid recognized overrides. Do not rely on an override to repair malformed input.

Then resolve each permitted field:

1. Apply mandatory provider policy. Reject customer settings that violate it.
2. Resolve approved customer settings and host settings into one trusted override set.
3. Use a trusted override when present. Otherwise, use the permitted model value.
4. Normalize the query. Derive locale and domain fields only when no explicit setting blocks that mapping.
5. Apply defaults to fields still unset. A fallback language: "EN" must not suppress lang:fr.
6. Validate the resulting field combinations. Build the API request only after these checks pass.

Figure 1Magnify

Keep defaults distinct from overrides. A default allows a model value to replace it. An override does not.

For MCP, the wire precedence is fixed: X-Search-* headers override params._meta, which overrides model arguments. Merge fields individually. Avoid conflicting header and _meta values. The server does not distinguish a human setting from another host setting.

3.3 Provider metadata: Baseten example

The Baseten integration uses this discovery metadata:

{
  "co.baseten/llm-controllable-params-default": ["query", "freshness"]
}

This lists preferred model-controlled fields. It does not set search values. It is provider-specific metadata, not a standard MCP permission mechanism.

Your provider can agree on its own metadata key and preferred fields with You.com. You.com can publish that key in the tool definition. Alternatively, keep the configuration in your runtime. Do not assume the hosted server already publishes your key.

In either case, the host must restrict the schema sent to the model and validate generated arguments. A new namespace alone changes no behavior. Keep the You.com override contract under com.you.com/search unchanged.

3.4 Recommended integration defaults

These values define the baseline for this runbook. The Baseten example concerns model permissions, not a required configuration for other providers.

Freshness is not a strict intersection with dates in the query. The API uses the broader timeframe when both specify a time window.

For Contents, send formats: ["markdown"]. The crawl timeout defaults to 10 seconds. Omit max_age to accept cached content regardless of age.

The public REST contract makes knowledge optional. Set "core" explicitly to guarantee this integration default. For MCP, check the connected server's schema before relying on its default.

4. Path A: hosted MCP

4.1 Connect with the provider key

Use Streamable HTTP and an MCP client that handles initialization and protocol negotiation.

https://api.you.com/mcp?tools=you-search,you-contents

Set this header in the server-side MCP client:

Authorization: Bearer <YDC_API_KEY>

Use the provider key for customer requests. Customer OAuth and unauthenticated profiles are outside this integration flow.

Initialize the connection, complete the MCP handshake, and call tools/list. Confirm that both tools are present. If the transport returns a session identifier, let the client manage it.

4.2 Read the runtime contract

For each tool, retain its description, inputSchema, and outputSchema. Preserve field descriptions, types, enum values, and constraints.

For you-search, also read tool._meta["com.you.com/search"]. It publishes the accepted override schema. Read a provider-specific metadata key only if it is present and your integration recognizes it.

Cache the discovered contract for the connection or release you tested. Refresh it after a server change or a tool-list change notification. Do not cache an unknown schema indefinitely.

4.3 Restrict model input and apply host values

The host does not need a separate MCP proxy to restrict the model. It can filter the schema before model inference and validate the resulting call before execution.

1. Expose the permitted fields, such as query and freshness.
2. Preserve their descriptions. Do not replace them with field names alone.
3. Reject fields that the model is not permitted to set.
4. Add trusted settings through params._meta["com.you.com/search"].
5. For a model-controlled hybrid field, apply a provider default only when the model omitted it.

For a soft default, the host can fill an omitted MCP argument. For a fixed value, use an override. Do not inject the language fallback as an explicit override unless you intend to suppress lang: mapping.

No supported false, null, or "off" value disables knowledge in the MCP contract described here. Omitting it does not disable a server default. Do not invent an opt-out value.

4.4 Send calls

This example pins result volume and extraction. It explicitly enables knowledge. It leaves locale available to operator mapping.

{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/call",
  "params": {
    "name": "you-search",
    "arguments": {
      "query": "EV tax incentives",
      "freshness": "month"
    },
    "_meta": {
      "com.you.com/search": {
        "knowledge": "core",
        "count": 30,
        "extraction": "highlights",
        "safesearch": "strict"
      }
    }
  }
}

After source selection, read URLs with you-contents:

{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "you-contents",
    "arguments": {
      "urls": ["https://you.com/docs/api-reference/search/v1-search"],
      "formats": ["markdown"]
    }
  }
}

These are MCP messages, not REST request bodies. The search override namespace does not configure Contents. Apply Contents policy in the host before the call.

4.5 Search override headers

Use either per-call _meta or these MCP HTTP headers. Prefer _meta for customer-specific settings. Headers win when both supply the same field.

Send numbers as header strings and domain lists as comma-separated strings. In _meta, use JSON numbers and arrays. Use _meta for large lists to avoid HTTP header limits.

Unknown override keys are ignored by the server. Invalid values on recognized keys produce errors. Validate in the host so a misspelled key cannot silently bypass policy.

These headers are MCP controls. For direct REST, resolve the settings in your host and put the result in the JSON body.

4.6 MCP defaults and response handling

Use this effective configuration for the baseline integration:

On success, prefer structuredContent:

• Search returns the API object with results and metadata.
• Contents returns { "output": [...] }.
• The first text block also contains the JSON representation. Do not add both copies to model context.
• Empty Search results can include a second text block with recovery guidance. Preserve that guidance separately.
• Errors can set isError: true without structuredContent. Do not parse error prose as JSON.

The optional _meta["you/usage"] describes retrieval activity. It is not an invoice. Some fields apply only to full-page extraction. pages_extracted does not establish how many pages were billed or read by the model.

4.7 Query normalization contract

MCP performs this transformation. A direct REST adapter must implement it if it promises the same model-facing query syntax. Do not normalize twice on the MCP path.

Process tokens in order. Preserve double-quoted groups, including groups inside operators such as intitle:"annual report". Apostrophes do not delimit groups.

Apply these rules:

The source settings and the defaults are different. For locale, use explicit setting, then derived operator value, then fallback. An explicit empty include_domains list suppresses domain extraction. An empty exclude_domains list does not. Omit unused lists rather than sending placeholders.

After normalization:

1. Reject an empty query after trimming.
2. Reject more than 400 characters or 50 whitespace-separated words. For MCP parity, the character count uses JavaScript string length.
3. Reject derived or explicit lists that exceed the supported path limit.
4. Reject non-empty include_domains combined with non-empty exclude_domains or boost_domains.
5. Apply the extraction checks in section 6.3.

The diagram shows the token loop and its validation checks:

Figure 2Magnify

Examples, with no explicit settings unless shown:

Mapping a URL to a domain does not retrieve that page. If the user asks to read a known URL, call Contents instead.

5. Path B: direct REST

5.1 Endpoints and authentication

Send X-API-Key: <YDC_API_KEY> and Content-Type: application/json. Use POST for new integrations. Do not send MCP envelopes, _meta, or MCP control headers to these endpoints.

5.2 Build model and API schemas from public contracts

Use these public sources:

• Search request and response reference
• Contents request and response reference
• Search OpenAPI
• Contents OpenAPI

Keep two contracts: the small model tool schema and the complete REST request/response schema. They are not interchangeable.

Preserve descriptions, types, enum values, constraints, optional fields, and nullable fields from the public references. Add this runbook's model guidance. Apply provider defaults in code rather than copying REST defaults into the model tool.

A minimal model search schema can use:

{
  "type": "object",
  "properties": {
    "query": {
      "type": "string",
      "minLength": 1,
      "description": "Find one thing with a short natural-language phrase. The host maps site:, lang:, and loc:. Maximum 50 words and 400 characters after mapping."
    },
    "freshness": {
      "type": "string",
      "description": "Set for time-sensitive facts. Use day, week, month, year, or YYYY-MM-DDtoYYYY-MM-DD. Omit for evergreen facts. The broader window wins when query text also specifies a timeframe."
    }
  },
  "required": ["query"],
  "additionalProperties": false
}

Describe the tool's purpose too: Search finds sources; Contents reads known URLs. For Contents, require a non-empty URL list in your model tool. Preserve the descriptions for formats, crawl_timeout, and max_age if you expose them.

Validate output with the REST response schema. Preserve URLs, content fields, dates, and knowledge attribution. Give the model field descriptions through the tool output schema or documented result context. Do not reduce all output to an unlabeled text string.

MCP tools/list publishes its own descriptions and schemas. These are not identical to the REST schemas. If your inference API uses a restricted JSON Schema dialect, adapt the schema without changing the intended behavior.

5.3 Map the effective fields to REST

REST also supports extraction.highlights.total_max_tokens. Its range is 1024–8192, with a default budget of 8192. It limits approximate highlight tokens across web results, not total response bytes. Treat it as a host setting. Do not send it through MCP unless the discovered contract supports it.

The current REST reference permits 5000 excluded domains; the documented MCP contract permits 500. Use 500 for a shared compatibility profile. Inclusion and boost lists permit 500 on both paths. Check the contract when you upgrade.

5.4 Search fan-out: what to avoid and what to send

Do not use full-page extraction for unrestricted parallel search. This is a valid REST body, but a poor default for every branch:

{
  "query": "EV tax incentives",
  "count": 10,
  "knowledge": "core",
  "language": "EN",
  "extraction": {
    "extraction_mode": "full_page",
    "extraction_source": "fetch",
    "full_page": { "extraction_formats": ["markdown"] }
  },
  "crawl_timeout": 10
}

With five queries and count: 10, the upper bound is 100 web/news result pages. These need not be distinct URLs. Full-page extraction happens before the host ranks or deduplicates sources. It can increase crawl cost, latency, response bytes, and model input tokens.

The 1.7 MB observation in section 1 shows the failure risk; it is not a prediction for this example. Check each response limit and the combined context limit. Limiting concurrency alone does not limit total context.

extraction_source: "cache" avoids live crawling but does not cap payload size. A cached page can still be large. A low count also does not guarantee a safe payload.

Use highlights for discovery, then Contents for selected pages. This REST example lowers the baseline count for fan-out and sets an approximate highlight budget:

curl --fail-with-body -X POST https://ydc-index.io/v1/search \
  -H "X-API-Key: $YDC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "EV tax incentives",
    "count": 10,
    "freshness": "month",
    "knowledge": "core",
    "language": "EN",
    "extraction": {
      "extraction_mode": "highlights",
      "highlights": { "total_max_tokens": 2048 }
    }
  }'

The count and token budget are example host settings, not universal defaults. Omit crawl_timeout and extraction_source for highlights.

Figure 3Magnify

Reserve budget before dispatch. Count each parallel search, each retry, and each requested Contents URL. Apply limits per task and per customer.

For MCP fan-out, pin count and extraction: "highlights" through the search override contract. Apply host payload limits even when a server token control is unavailable.

Use full-page Search only for a workload that needs the returned pages and has explicit page, concurrency, byte, and token budgets. For known selected URLs, use Contents.

5.5 Request algorithm

1. Validate model input against the restricted tool schema.
2. Validate trusted settings and enforce provider policy.
3. Resolve explicit overrides over permitted model values.
4. Normalize query operators using section 4.7.
5. Fill unset defaults from section 3.4. Do not replace explicit values.
6. Apply the query, domain, and extraction checks in section 6.
7. Reserve task and customer budget before dispatch.
8. Map fields to the REST body. Omit unused fields.
9. Send POST with a bounded request deadline.
10. Check HTTP status before parsing a success response.
11. Validate the response. Record request identifiers and retrieval usage.
12. Apply response and model-context limits. Preserve source attribution.
13. Return structured evidence or an actionable error to the model.

5.6 Search response handling

The REST response is an object containing results and metadata. Each result section can be absent. Use empty arrays for counting when a section is absent; do not fabricate result records.

Knowledge attribution names data providers; it is not a URL citation. Do not invent URLs for these facts or submit them to Contents. An absent knowledge section is valid. Ignore unsupported knowledge types without failing the entire search.

REST does not return MCP structuredContent or _meta["you/usage"]. Compute host metrics separately. results_returned is the sum of web, news, and knowledge counts. For parity with MCP full-page metrics, count web items with returned Markdown or HTML as pages_extracted.

Keep retrieval metrics distinct from billing and model-read metrics. Record requested URLs, returned URLs, bytes, latency, and truncation separately.

5.7 Contents request and partial results

Send only selected public HTTP or HTTPS URLs. A user-supplied URL can go directly to Contents; a search is not required.

curl --fail-with-body -X POST https://ydc-index.io/v1/contents \
  -H "X-API-Key: $YDC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://you.com/docs/api-reference/search/v1-search"],
    "formats": ["markdown"],
    "crawl_timeout": 10
  }'

• Use a non-empty URL list. Select Markdown for model reading; add HTML or metadata only when needed.
• Contents crawl_timeout is independent of Search extraction. Use an integer from 1 to 60 seconds.
• max_age is nullable or a non-negative integer in seconds. Omit it for no cache-age limit. Use 0 to force a fresh fetch.
• Search freshness does not control the Contents cache.
• Start with small URL batches. A batch limit of 10 is a provider policy example, not a limit declared by the current Contents OpenAPI schema.
• Discover the MCP URL limit from tools/list. Do not assume its limit proves the REST service limit.

REST returns an array; MCP wraps that array in output. Match entries by URL, not array position. A multi-URL response can omit a PDF that is too long to process. A single-URL request for that PDF can return 422.

Treat missing or empty requested text as an unread page, not as evidence. A metadata-only request can succeed without Markdown or HTML. Preserve partial successes and report missing URLs. Count URLs for usage; do not equate one batch with one billable page.

6. Runtime controls

6.1 Query and schema checks

• Require a non-empty query.
• Enforce 400 characters and 50 words after normalization.
• Validate all explicit enum values using the selected public or discovered contract.
• Reject unknown model fields. Detect unknown host setting names before execution.
• Apply defaults only to missing fields. Do not treat 0, an empty array, and an absent value as interchangeable.
• Preserve unknown query operators; do not claim the engine supports them.

6.2 Domain policy

Reject conflicting non-empty inclusion/exclusion or inclusion/boost lists. Exclusion and boost can be combined. Do not silently truncate domain lists.

Operator normalization is not a security control. The public REST strict_operators option has no effect with highlights. Do not use an inline site: filter as a substitute for mandatory source policy.

Check returned URLs against provider policy before Contents calls or model exposure. Validate public HTTP/HTTPS URLs. Block private, loopback, and link-local targets. Apply equivalent checks to redirects and DNS resolution if your host fetches pages itself.

6.3 Extraction checks

• Allow count from 1 to 100 generally; limit it to 50 with highlights.
• For REST highlights, omit crawl_timeout and extraction_source. Both combinations can produce 422.
• For full-page, nest extraction_source under extraction. Keep crawl_timeout at the top level.
• To match MCP, drop extraction_source when a host override changes extraction to none or highlights.
• Otherwise, reject extraction_source without full-page extraction.
• The MCP adapter omits Search crawl_timeout for non-full-page modes. Reproduce this when you accept the same flat tool fields.
• Enforce byte and token limits on both Search and Contents. A count or timeout is not a size limit.

6.4 Zero-result recovery

A successful empty response is not an HTTP failure. Count web, news, and knowledge results before declaring it empty.

Allow one revised query within the task budget:

1. Remove exact-phrase constraints first, if they are not required by the task.
2. Use more common terms or another aspect of the question.
3. Keep required domain policy. Do not broaden access to rescue a query.
4. Read a useful prior source instead, when one exists.
5. If evidence remains incomplete, answer the supported part and state the gap.

Do not resend the same empty query automatically. Keep recovery guidance outside the API response schema in a host message or a separate tool text block.

6.5 Read discipline without SystemOne

Use this baseline when no separate judge is available. Code owns the read queue and budget; the main model can propose sources.

1. Search with highlights. If the user already supplied a URL, start with Contents.
2. Apply source policy and deduplicate URLs across all search branches.
3. Use task requirements to decide what needs page text. Quotes, tables, comparisons, and detailed page claims need the relevant source content.
4. Read a small number of selected sources. One or two pages is a starting policy, not an API requirement.
5. Check that the requested content arrived. Do not count a failed or metadata-only fetch as a page read.
6. Generate only claims supported by the evidence. Preserve source links and any limitations.

Highlights can support a focused fact when the excerpt contains sufficient evidence. Do not use a blanket rule that summaries may rely on snippets. Do not claim to have read a page when only an excerpt is available.

Define a read miss as a task requiring page evidence with no successful relevant page read. If the model tries to answer in that state, add a targeted read instruction or schedule the read in code.

Do not force a new Contents call for every search. Reuse already-read evidence when policy and freshness permit. Enforce separate search-call, URL-read, byte, and token budgets.

6.6 Read discipline with SystemOne

Optionally add a small typed judge for fast query checks and routing decisions. SystemOne (Jev) is one implementation. It is not required by You.com.

The host still performs syntax validation, normalization, permissions, and budget checks. Do not delegate these deterministic checks to the judge.

Figure 4Magnify

Batch independent judgments when supported. Bound candidates, excerpt lengths, judge input tokens, and judge latency. Rank candidates and execute only the highest-ranked queries within the host budget.

A score threshold is optional host policy. Calibrate it on your tasks before using it to reject queries or sources. Do not let the judge override mandatory source rules or skip required evidence.

Validate every judge output. On timeout or invalid output, use the deterministic discipline in section 6.5, or stop according to policy. Do not remove budgets during fallback.

The public risk-analysis-server pattern demonstrates proposal ranking, result scoring, selected-page retrieval, and a replaceable judge. It uses a severity choice for its risk task. The generic routing labels above are a recommendation, not a claim about that implementation.

6.7 Errors, retries, and diagnostics

For MCP, check both transport errors and tool isError. A successful HTTP response can contain a failed tool call.

Retries can add cost. Count every attempt. Set request deadlines independently of crawl timeouts. Do not turn an exhausted retry loop into another unbounded agent loop.

Log a provider request ID, customer identifier, schema version, effective settings, status, latency, response size, and available You.com request identifiers. Redact keys and sensitive query data. Send sanitized errors to the model.

Treat retrieved content as untrusted evidence, never as instructions. Retain URLs and provenance when reducing context. Mark truncated content; do not present it as the full page.

7. Integration decisions to record

Use your existing configuration system. You do not need a new provider identity object or a special file format.

Record these decisions for implementation and review:

Separate API limits from your policy. For example, 10 Contents URLs per batch or 10 total tool calls can be starting policies. They are not universal You.com limits.

If you support both transports, test the same effective settings and model-visible evidence on both paths. Document deliberate REST-only features rather than claiming exact parity.

8. Acceptance tests

Use fixtures for failure and payload tests. Use a provider test key for live smoke tests. Do not use production customer prompts or spend on an uncontrolled full-page fan-out test.

8.1 Contract and control tests

• MCP initializes and discovers Search and Contents. A missing optional provider metadata key does not break the integration.
• Model input contains only permitted fields and retains their descriptions.
• Trusted settings never come from model-generated _meta or arbitrary customer headers.
• Defaults fill omissions; they do not overwrite valid model values.
• MCP headers override _meta per field. Both override model arguments for supported fields.
• Invalid arguments and recognized invalid overrides fail before an API call.
• Knowledge is enabled as the string "core", including when the model omits it. Boolean true is not sent.
• A customer cannot weaken mandatory provider source or moderation policy.

8.2 Request and normalization tests

• A baseline request sends count: 30, knowledge: "core", language fallback EN, and highlights in the correct transport shape.
• The normalization examples in section 4.7 produce the stated outputs.
• lang:fr beats the fallback EN, but not an explicit language override.
• Empty normalized queries, more than 400 characters, and more than 50 words fail locally.
• Inclusion/exclusion and inclusion/boost conflicts fail. Exclusion/boost succeeds within supported limits.
• A list above the configured path limit fails without truncation.
• count: 51 with highlights fails before dispatch.
• Highlights omit Search crawl_timeout and extraction_source.
• Full-page mode nests extraction_source correctly and keeps crawl_timeout at the top level.
• A host extraction downgrade clears an otherwise incompatible extraction_source.
• none omits REST extraction rather than sending an unsupported mode.

8.3 Response and evidence tests

• Missing result sections do not fail parsing. Knowledge-only results do not trigger zero-result recovery.
• Knowledge attribution and as_of survive formatting; no citation URL is invented.
• MCP structured and text JSON copies are not duplicated in model context.
• MCP error prose is not parsed as a success object.
• Contents results are matched by URL. Missing entries and null or empty text do not count as successful reads.
• Metadata-only success is not mislabeled as extraction failure.
• Recovery guidance causes at most one revised query and preserves mandatory domain policy.
• Both read disciplines preserve evidence requirements and return a partial answer when budgets are exhausted.

8.4 Operational tests

• Parallel branches reserve one shared task budget before dispatch.
• Fan-out uses highlights; selected URLs are deduplicated before Contents.
• Oversized single responses and combined context overflow are handled before model submission.
• Cached full-page fixtures still trigger size checks.
• Retries obey deadlines, backoff, and attempt limits. 401, 402, and 403 do not trigger retry loops.
• Judge timeout or invalid output follows the configured fallback without relaxing policy.
• One customer's settings, cache entries, and rate limits cannot affect another customer's request incorrectly.
• A live smoke test completes Search, selected Contents, and a source-linked answer on each enabled path.

9. Production checklist

• ☐Search and Contents access is confirmed for the provider key.
• ☐The key remains server-side. Customer quotas protect shared capacity.
• ☐Model permissions, defaults, overrides, and normalization are implemented separately.
• ☐Knowledge uses "core"; Search uses highlights by default.
• ☐Public REST schemas or discovered MCP schemas match the tested runtime.
• ☐Fan-out has shared query, read, concurrency, byte, and token budgets.
• ☐Contents partial failures do not become fabricated evidence.
• ☐The selected read discipline enforces required source reading.
• ☐Retry, timeout, and optional judge failure paths are bounded.
• ☐Logs contain useful request identifiers but no API keys.
• ☐Retrieved instructions cannot override host policy.
• ☐Acceptance tests and a source-linked smoke test pass.

10. Public references

• MCP server and search parameter control
• Search API reference
• Contents API reference
• Search OpenAPI schema
• Contents OpenAPI schema
• Combined OpenAPI
• Search request controls
• Search operators
• Highlights and full-page extraction
• API billing
• Risk-analysis-server reference pattern