SERP Scraping API vs Web Search API: Which One Do You Need?

TLDR: Choose a SERP scraping API when your product needs Google's results page itself: rank positions, ads, local results, and SERP features for a given city and device. Choose a web search API when your code needs documents to read, such as RAG context or agent evidence: it returns the provider's own ranking as JSON, and some providers attach page text. Products with both needs run both, split by output.

For how each category works, see what a SERP API is and what a web search API is. This guide is about choosing between them.

What is the real difference between the two?

Both categories take a query and return JSON. What differs is whose ranking you receive and what the payload contains.

A SERP scraping API sends your query to a search engine, usually Google, from the vendor's own infrastructure, then parses the results page into fields. You get Google's order and page features for the location and device you request; SerpApi's Google Search API reference, for example, documents location and device parameters.

A web search API queries an index the provider crawls and ranks itself. Brave, for example, says its API is powered by its own index and is "not a scraper" that queries Google or Bing (Brave Search API). The You.com Web Search API returns web and news results with URLs, titles, snippets, and dates, and can return query-relevant highlights or full-page Markdown in the same response.

The useful question is which ranking and which payload your product consumes. If you need Google's positions, only a SERP API reports them. If you need text a model can cite, a SERP API gives you links and snippets, and the text becomes a second job. The web search API guide covers how index owners and scrapers differ in more depth.

How do they compare on the criteria that decide it?

The table uses documented behavior from representative vendors, as of September 2026. Two rows usually settle the choice on their own: whose ranking you need, and whether you need page text.

Sources: SerpApi reference, SerpApi pricing, DataForSEO pricing, Bright Data pricing, You.com request controls, billing, and rate limits, Brave Search API, and Exa pricing.

SerpApi can also return the results page as Markdown (output=md). That helps a model read the SERP, but it is still the results page, not the documents behind it.

Which one fits your workload?

The rule of thumb: anything measured against Google's page goes to a SERP API, and anything that reads documents goes to a web search API. A job with both outputs needs both, with two budgets. Competitive research can land on either side, depending on whether you need to know where a competitor ranks or what its pages say.

What does each option cost at realistic volume?

These list prices come from each vendor's pricing page as of September 2026. They change often, so reprice before you sign.

Worked example: daily rank tracking

Tracking 2,000 keywords daily in one market is 60,000 checks a month.

• Top 10 only. DataForSEO's standard queue costs 60,000 × $0.0006 = $36 a month, or $120 in live mode. Bright Data pay as you go costs $90. On SerpApi, the smallest plan that covers 60,000 searches is Searcher: $725 for 100,000.
• Top 100. Since September 19, 2025, DataForSEO bills each page of 10 results separately, and its pricing FAQ names a Google update as the catalyst. A top-100 check is 10 pages, so the month becomes 600,000 SERPs: $360 in the standard queue or $1,200 live. The same FAQ says deep fetches take 3 to 4 times longer than page one. SerpApi's reference paginates with start in steps of 10, so 100 positions can take 10 searches per keyword, which lands on the $3,750 Cloud 1M plan.

A web search API cannot do this job at any price, because it does not report Google positions.

Worked example: agent grounding

An agent runs 100,000 searches a month and needs readable text from about five pages per search.

• You.com with highlights: 100,000 calls × $5 per 1,000 = $500. Highlights are part of the call price.
• You.com with full pages: with count set to 5, a full-page call can return up to 10 pages (5 web and 5 news). If every page is crawled live, that adds $1,000. The default blend source serves cached pages free, so the month lands between $500 and $1,500.
• Brave or Exa, search only: $500 on Brave (less the $5 monthly credit) or $700 on Exa at 10 results, plus extraction if you need full pages.
• SERP API plus extraction: DataForSEO live mode costs $200 (its queues are too slow for an interactive agent), or SerpApi Searcher $725. Add 500,000 page extractions: at the $1.00 per 1,000 pages that You.com and Exa list for their contents APIs, that is $500, for $700 to $1,225 in total. Otherwise you run your own fetchers, proxies, and headless browsers.

At list price the grounding totals overlap. What separates them is hops and failure domains: the SERP path is a search call plus a fetch per page, and each fetch can be blocked, time out, or return boilerplate. The extraction pipeline guide covers what that second stage involves. These totals also leave out unused plan capacity (SerpApi tiers are fixed), credit expiry (Serper credits last six months), retries, and engineering time. Free allowances help at small scale: SerpApi includes 250 searches a month, Brave $5 a month, and new You.com accounts start with $100 in credits.

What are the platform risks of scraping Google?

SERP scraping depends on access that Google actively resists. Google's complaint says it launched a protection system called SearchGuard in January 2025 to block automated access to its results. On December 19, 2025, Google sued SerpApi under the DMCA's anti-circumvention provisions. On July 20, 2026, the court dismissed the original complaint, partly without leave to amend. Google filed an amended complaint in August 2026, and SerpApi's motion to dismiss it was pending when this guide was updated in September 2026, according to the public docket. The case is unresolved and names one vendor, but it shows that the access every Google SERP API relies on is contested.

The official routes are closing at the same time. Google's Custom Search JSON API is closed to new customers, existing customers have until January 1, 2027 to move, and full-web users are asked to contact Google. Microsoft retired the Bing Search APIs on August 11, 2025 and points customers to Grounding with Bing Search in Azure AI Agents. For a new project, neither company offers a self-serve API for raw full-web results. If you are migrating, see the Google CSE migration guide and the Bing Search API alternatives.

None of this makes SERP APIs unusable. For rank tracking there is no substitute, and vendors offer mitigations: SerpApi's U.S. Legal Shield provides up to $2 million in coverage for scraping and parsing search engine data on its Production plan and above. Put the provider behind an interface, keep a second vendor tested, store the raw responses you are allowed to keep, and have counsel read the terms against your use. Independent-index APIs carry a different risk: their ranking and coverage differ from Google's, which matters only when Google's order is what you need.

When should you run both?

Run both when one product has two outputs. An SEO platform is a typical example: nightly rank checks go to a SERP API's cheap queue, while the assistant that drafts content briefs needs page text on demand. Split those branches early, give each its own cache policy and timeout, and normalize results into one record so downstream code does not care which path produced a row.

The script below does that for a side-by-side test. It maps a SerpApi Google response and a You.com Web Search API response into the same record and counts rows that arrive with text a model can use. It runs offline against small fixtures by default. With --live, it makes one call to each API and prints counts and wall time, never keys or content.

"""Same query, two paths: a SERP API call and a web search API call.

Offline by default: parses small fixtures and checks the mapping.
With --live it calls SerpApi and You.com once each, reading keys from
SERPAPI_API_KEY and YDC_API_KEY. It prints counts and timing, never keys or content.
"""
import argparse
import json
import os
import time
import urllib.error
import urllib.parse
import urllib.request

SERPAPI_URL = "https://serpapi.com/search.json"
YOU_URL = "https://ydc-index.io/v1/search"


def from_serpapi(payload):
    """Google's organic order as SerpApi parses it; evidence is Google's snippet."""
    rows = []
    for hit in payload.get("organic_results") or []:
        snippet = hit.get("snippet")
        rows.append({"ranking": "google", "rank": hit.get("position"),
                     "url": hit.get("link"), "evidence": [snippet] if snippet else []})
    return rows


def from_you(payload):
    """The provider's own order; evidence is highlights if requested, else snippets."""
    web = (payload.get("results") or {}).get("web") or []
    rows = []
    for rank, hit in enumerate(web, start=1):
        contents = hit.get("contents") or {}
        evidence = contents.get("highlights") or hit.get("snippets") or []
        rows.append({"ranking": "you.com index", "rank": rank,
                     "url": hit.get("url"), "evidence": evidence})
    return rows


def summarize(rows):
    return {"rows": len(rows),
            "with_text": sum(1 for r in rows if r["evidence"]),
            "evidence_chars": sum(len(" ".join(r["evidence"])) for r in rows)}


def probe(request, timeout, parse):
    start = time.monotonic()
    try:
        with urllib.request.urlopen(request, timeout=timeout) as resp:
            payload = json.load(resp)
    except urllib.error.HTTPError as exc:
        return {"http_error": exc.code}
    wall_s = round(time.monotonic() - start, 2)
    return dict(summarize(parse(payload)), wall_s=wall_s)


def live(query):
    serp_key = os.environ.get("SERPAPI_API_KEY")
    you_key = os.environ.get("YDC_API_KEY")
    if not serp_key or not you_key:
        raise SystemExit("Set SERPAPI_API_KEY and YDC_API_KEY first")
    params = urllib.parse.urlencode({
        "engine": "google", "q": query, "api_key": serp_key,
        "location": "Austin, Texas, United States", "device": "desktop"})
    body = json.dumps({"query": query, "count": 10, "country": "US",
                       "extraction": {"extraction_mode": "highlights"}}).encode("utf-8")
    you_request = urllib.request.Request(YOU_URL, data=body, method="POST", headers={
        "X-API-Key": you_key, "Content-Type": "application/json"})
    return {"serpapi": probe(SERPAPI_URL + "?" + params, 60, from_serpapi),
            "you": probe(you_request, 30, from_you)}


SERP_FIXTURE = {"organic_results": [
    {"position": 1, "link": "https://a.example", "snippet": "Short excerpt."},
    {"position": 2, "link": "https://b.example"}]}
YOU_FIXTURE = {"results": {"web": [
    {"url": "https://c.example", "contents": {"highlights": ["Passage one.", "Passage two."]}},
    {"url": "https://d.example", "snippets": ["Snippet."]},
    {"url": "https://e.example"}]}, "metadata": {"latency": 0.41}}


def offline():
    serp, you = from_serpapi(SERP_FIXTURE), from_you(YOU_FIXTURE)
    assert [r["rank"] for r in serp] == [1, 2] and serp[1]["evidence"] == []
    assert you[0]["evidence"] == ["Passage one.", "Passage two."]
    assert you[1]["evidence"] == ["Snippet."] and you[2]["evidence"] == []
    assert from_you({}) == [] and from_serpapi({}) == []
    return {"serpapi": summarize(serp), "you": summarize(you)}


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--live", action="store_true")
    parser.add_argument("--query", default="crm for small business")
    args = parser.parse_args()
    print(json.dumps(live(args.query) if args.live else offline(), indent=2))

Read with_text as the number of rows your model can use without another fetch. SerpApi rows carry Google's snippet; You.com rows requested with highlights carry query-relevant passages, and evidence_chars shows how much text each path returned. One wall_s sample means little, so repeat the run at production concurrency before trusting it.

How do you test the choice before committing?

1. Name the output. Write down what your code consumes after the call: positions and SERP features, or passages and pages. If the list has both, budget for both.
2. Use real queries. Pull 100 to 200 from your logs, including long-tail and local ones, and fix location and language for the whole run.
3. Measure tails, not one call. Log p50 and p95 wall time at your expected concurrency. A tail-latency view shows what users actually wait for.
4. Count the hops. Every fetch-and-extract step after search is a failure domain with its own timeouts, blocks, and cleanup.
5. Price the successful unit. Work out cost per tracked position or per grounded answer, including extra pages, retries, extraction, and unused plan capacity.
6. Force failures. Trigger 429s, empty results, and blocked pages, and check how each vendor reports them. You.com returns a 429 (with a Retry-After header when present) and sends rate-limit headers on every response.

The web search API evaluation guide has a fuller protocol for the retrieval side. If the test shows you need Google's page, choose a SERP vendor on location accuracy, parser upkeep, and queue pricing. If it shows you need documents, choose a web search API on evidence per call and tail latency.