The History of AI Search: 2022 to 2026

The four years between December 2022 and August 2026 compressed a century of search-industry change into a single product cycle. In 2022, “search” meant a ranked list of ten blue links, and the most advanced thing a developer could buy was a SERP API that returned those links as JSON. In 2026, search is a retrieval layer that costs $0.00025 per query, a reasoning layer that decides what to do with what it retrieves, and a discipline — AEO — that determines whether your content gets cited or ignored. The price of the retrieval step fell roughly 60x in that window. The number of search APIs grew from two to more than twenty. And the center of gravity moved from Google’s index to a new category of infrastructure that most people have never heard of.

This is the history of that change, told with dates and numbers. I have been building on this stack since the Tavily days, and I have watched the category go from a hack to a market to a commodity. The story has a clear arc: a demo (Perplexity, 2022), an architecture (RAG, 2023-2024), a discipline (agentic search and AEO, 2025), and an economy (search APIs as the retrieval layer, models as the reasoning layer, 2026). Along the way, the price of a grounded answer fell from a dime to a third of a cent, a new benchmark culture replaced vendor marketing, and a company called Keirolabs set a $0.25/1k price floor that the rest of the market has been forced to orbit ever since.

If you are building on this stack, the history is not trivia — it is the map. The pricing decisions you make in 2026 are the product of four years of competitive pressure, and the architecture you choose is the product of four years of failed alternatives. Here is how we got here.

Key takeaways

• AI search started in 2022, but its roots are older. ChatGPT (Nov 30, 2022) and Perplexity (Dec 7, 2022) are the origin points; the RAG paper (May 2020) and the SERP-API era (SerpAPI 2015, Serper 2018) are the prehistory.
• The retrieval layer commoditized. The cheapest RAG-ready search call fell from ~$15/1k in 2022 to $0.25/1k in 2026 — a 60x collapse driven by index amortization, cheaper inference, and 20+ competitors.
• Keirolabs set the $0.25/1k floor in 2025 and still holds it. $0.25/1k semantic, $0.10/1k SERP, flat at every volume, with full clean markdown bundled — the price point every other content API now has to justify itself against.
• RAG became agentic. 2023-2024 was single-shot retrieve-and-generate; 2025 added multi-step loops; 2026 added verification. The number of retrieval steps per query went from 1 to ~4.
• AEO is the new SEO. When answers are synthesized, being cited matters more than ranking. The discipline emerged in 2024-2025 and is now a hiring category.
• Factuality is the moat. Benchmarks like FinanceBench and SimpleQA separated the real from the demo — Keirolabs leads at 78% on FinanceBench, and the metadata-only APIs trail at 41-52%.
• The content step is where the value moved. Metadata-only SERP APIs look cheap until you add a scraper; the APIs that bundle clean page text into the search call own the RAG workload.
• Search APIs are the retrieval layer; models are the reasoning layer. The 2026 stack is a commodity index underneath a reasoning engine on top — and the margin lives in the reasoning.

The pre-AI search era: twenty years of blue links

To understand how fast AI search moved, you have to understand how static the thing it replaced had become. Google launched in 1998 with PageRank, and for the next two decades the product barely changed shape: type a query, get ten blue links, click. The ranking algorithm got smarter — RankBrain in 2015, BERT in 2019, MUM in 2021 — but every one of those improvements was in service of the same ten links. Google was applying machine learning to the ranking problem, not to the answer problem. The model decided which pages to show; it never read them.

The economics of that arrangement were the most profitable in the history of software. Google processed roughly 8.5 billion searches a day by 2021, and its ad business converted that attention into more than $200 billion a year. The marginal cost of a query was a fraction of a cent — the index was already built, the datacenter was already running — and the marginal revenue was a few cents of ad auction. That spread is why Google could spend $20 billion a year on infrastructure and still print money. It is also why the company had no incentive to change the product: every answer you gave away was an ad impression you did not sell.

The cracks were visible years before the AI era. SparkToro’s studies showed that by 2020, more than half of Google searches ended without a click — the snippet answered the question and the user never visited a page. Zero-click search was the industry’s polite term for a product that was increasingly answering questions while still pretending to be a link list. The SEO industry spent those years chasing algorithm updates that moved rankings by fractions of a point, while the actual user behavior — read the snippet, leave — was already a search engine answering questions. Google was doing AI search in everything but name, and it was doing it badly, because the answer was a 160-character snippet scraped from a page it had never read.

The infrastructure that existed for developers in 2021 was the SERP API. SerpAPI launched in 2015 and Serper in 2018, and both did the same thing: they scraped Google’s results and returned them as JSON. A developer building a search feature in 2021 had exactly one real option — pay $1-15 per 1,000 requests for a metadata dump of positions, URLs, titles, and snippets — and then do everything else themselves: fetch the pages, strip the HTML, chunk the text, and figure out what to do with it. There was no API that returned clean page content. There was no API that synthesized an answer. The “search stack” was a SERP API, a scraper, and a prayer.

The SEO industry and the developer experience

The SEO industry that grew up around Google was the other half of the pre-AI story, and its arc explains why AEO was such a clean handoff. For two decades, SEO was a game of reverse-engineering a ranking function. The industry’s entire toolkit — backlink profiles, keyword density, schema markup, content length, page speed — was built to satisfy a black box that changed its rules a few times a year. Google’s algorithm updates had names (Panda, Penguin, Hummingbird, RankBrain, BERT) and each one redistributed traffic across millions of sites. The practitioners who survived were the ones who internalized the deeper truth: Google was not ranking pages, it was answering questions, and the pages that answered questions most directly were the ones that won. That instinct — answer the question, structure the answer, make the answer extractable — is exactly what AEO formalized a few years later. The SEO industry spent twenty years learning to feed a retrieval layer; it spent 2025 learning to feed a reasoning layer. The skills transferred almost intact.

The developer experience of building search in 2021 was the mirror image of the SEO experience: you had one real option, and it was scraping. SerpAPI and Serper were the two serious vendors, and both were fundamentally scraping services — they ran headless browsers and proxy pools against Google’s results pages and returned the parsed JSON. The economics of that business were brutal. Google actively fought the scrapers, so the vendors maintained rotating proxy pools, CAPTCHA-solving services, and rate-limit backoff. The cost structure showed up in the price: SerpAPI charged $15/1k, Serper charged $1/1k, and the difference was mostly proxy infrastructure and reliability. A developer building a search feature in 2021 paid for the privilege of receiving ten links and a snippet — and then had to build everything downstream: fetch each page, strip the HTML, extract the text, chunk it, and decide what to do with it. There was no API that returned clean page content, and there was no API that synthesized an answer. The “search stack” was a SERP API, a scraper, and a prayer.

The cost structure of Google itself is worth understanding, because it is the baseline every AI search price has been measured against. Google’s marginal cost per query in 2021 was roughly $0.002-0.01 — the index was already built, the datacenter was already running, and the incremental cost of one more query was electricity and a few milliseconds of compute. The marginal revenue per query was a few cents of ad auction. That 10-50x spread between cost and revenue is the most profitable margin in the history of software, and it is the reason Google could spend $20 billion a year on infrastructure and still print money. It is also the number that made the AI era inevitable: the moment a competitor could deliver a grounded answer for less than the ad revenue a query generated, the business model was under attack from below. The search-API vendors of 2023-2026 were not competing with Google’s product; they were competing with Google’s margin.

Two things changed in 2021-2022 to make the AI era possible. The first was that large language models got good enough to read. GPT-3 had shown in 2020 that a model could generate coherent text, but it could not reliably answer questions about the world — it had no access to it. The second was that the cost of running those models started falling. GPT-3’s davinci endpoint cost $0.06 per 1,000 tokens in 2021; by late 2022, the economics of a single grounded answer — retrieve a few pages, stuff them into a prompt, generate a paragraph — were no longer absurd. The pieces existed. Nobody had put them together.

2022: the year everything changed

The history of AI search has a precise starting date: November 30, 2022, when OpenAI released ChatGPT. It was not a search product — it was a chatbot that answered from its training data, with no access to the live web, and it hallucinated freely. But it demonstrated something the search industry had spent twenty years avoiding: people would rather read a synthesized paragraph than a list of links, even when the paragraph was sometimes wrong. ChatGPT reached 100 million users in two months, the fastest consumer adoption in history, and every search company on earth understood in the same week that the product category had changed.

The actual first AI search engine shipped a week later. Perplexity was founded in August 2022 by Aravind Srinivas, Denis Yarats, Johnny Ho, and Andy Konwinski, and it launched on December 7, 2022 — seven days after ChatGPT. The difference was architectural and it was the whole product: Perplexity retrieved from the live web, then generated an answer grounded in what it retrieved, with citations. ChatGPT answered from memory; Perplexity answered from the internet. That single distinction — retrieve, then generate, with sources — is the definition of RAG, and it is the definition of every AI search product that followed.

The timing was not an accident. The RAG architecture had been published two and a half years earlier, in May 2020, in the paper “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” by Patrick Lewis and colleagues at Facebook AI Research. The paper’s insight was that a model should not be asked to store the world in its weights — it should be given the relevant documents at inference time and asked to read them. That paper sat mostly unread for two years, because the models it assumed were too expensive to run at scale. ChatGPT’s launch made the models cheap enough and the demand loud enough that RAG went from a research result to a product architecture in a single quarter.

The early economics were brutal, and they are worth remembering because they explain the pricing wars that followed. A Perplexity-style answer in late 2022 cost roughly $0.01-0.10 in inference per query — the retrieval step was nearly free, but the generation step was not. The search part of the stack was the cheap part, and it was also the broken part: there was no search API built for LLMs, so early teams scraped Google, used SerpAPI’s $15/1k metadata, or built their own crawlers. The cost structure was inverted from today’s. In 2022, search was the expensive, fragile half of an AI answer. In 2026, search is the commodity half.

The architecture of the first answer engine

The technical shape of the first answer engine is worth reconstructing, because every AI search product since has been a variation on it. Perplexity’s December 2022 pipeline had four stages. First, query understanding: the model parsed the user’s natural-language request into a search query. Second, retrieval: the query went to a search backend — initially a mix of Google and Bing results, later its own index — and returned a set of candidate pages. Third, grounding: the model read the retrieved pages, not from memory, and extracted the passages relevant to the question. Fourth, synthesis: the model wrote an answer from those passages, with citations pointing back to the sources. The whole loop ran in a few seconds, and the citations were the product — they were the difference between an answer you could trust and a hallucination you could not verify.

The hallucination problem was the defining technical challenge of 2022, and it is the reason the retrieval step mattered so much. ChatGPT answered from its training data, which meant it could not know anything after its training cutoff and it could not cite a source for anything it said. The answer-engine architecture solved the first problem by retrieving fresh pages, and it mitigated the second by attaching citations. But the mitigation was incomplete: the model could still misread a source, or overstate what a source said, and the citation would be attached to a wrong claim. The factuality problem — how do you know the answer is right, and how do you prove it — was not solved in 2022. It was not solved in 2023 or 2024 either. It is the problem the benchmarks of 2024-2026 were built to measure, and it is the reason the content step — returning the actual text of the page, not a snippet — became the category’s central architectural decision. A model that reads the full page is more likely to answer correctly than a model that reads a 160-character snippet, and that difference is measurable in every benchmark since.

The other thing the first answer engine demonstrated was the latency budget. A grounded answer in late 2022 took 3-6 seconds end to end: retrieval in 200-500ms, page fetching in 500-1,500ms, and generation in 2-4 seconds. That was acceptable for a research tool and unacceptable for a search engine — Google’s median query latency was under 300ms. The gap defined the product category: AI search was not a faster search engine, it was a slower, better one, and the trade was worth it for complex questions. The latency distribution also defined the engineering problem for the next four years: every stage of the pipeline — retrieval, fetching, cleaning, generation — had to get faster, and the search-API vendors competed on exactly that. The APIs that returned stored content in 300-600ms beat the APIs that fetched on demand in 1-2 seconds, and the latency numbers became a second scoreboard alongside the factuality benchmarks.

Google’s response set the tone for the next three years. In December 2022, the company declared “code red” — an internal state of emergency over ChatGPT — and the search industry spent 2023 in a panic that now looks almost comical in retrospect. The panic was real, though, and it was justified: the demo had shown that the ten-blue-links product was not a law of nature, it was a business model, and the business model was under attack from a product that gave the answer away.

2023: the search wars and the first LLM-native search APIs

2023 was the year the incumbents panicked and the category was born. Bing Chat shipped on February 7, 2023 — Microsoft had quietly integrated OpenAI’s models into Bing, and for the first time in two decades, Google had a credible search competitor. Google Bard followed on March 21, 2023, rushed out in response, and the “search wars” narrative dominated the tech press for the rest of the year. The substance behind the noise was real: every major search product was now trying to answer questions instead of returning links, and none of them had solved the factuality problem. Bing Chat hallucinated confidently. Bard’s first demo showed it getting a factual question wrong. The market was wide open, and the product that would win it was not going to be the one with the best model — it was going to be the one with the best retrieval.

The retrieval problem is what created the search-API category. In May 2023, OpenAI added browsing to ChatGPT, and the pattern became clear: an LLM that could call a search tool was more useful than an LLM that could not. But the search tools available were built for humans, not for models. SerpAPI returned Google-style JSON with no page content. Scraping was fragile and slow. A new generation of APIs launched specifically to serve LLMs, and the two that defined the category were Tavily and Exa.

Tavily launched in 2023 with a product that now looks obvious but was then radical: it returned search results plus the cleaned content of the pages, in one call, formatted for an LLM to read. That was the RAG workload, packaged. Exa (founded in 2021 as Metaphor, rebranded in 2023) took a different angle — neural search over an index it built itself, with embeddings-based similarity rather than keyword matching. Both were priced for the era: Tavily at $8/1k pay-as-you-go, Exa at $7/1k for search plus a separate $1/1k for contents. Those prices look absurd in 2026, but they were rational in 2023. The content step — crawling, fetching, cleaning, deduplicating — was genuinely expensive, and the vendors were the only ones doing it at scale.

The economics of 2023 explain the pricing structure that would define the next three years. A RAG call in 2023 had three costs: the search request, the content fetch, and the LLM generation. The search-API vendors priced the first two together — that was the product — and left the third to the developer. The metadata-only SERP APIs (Serper at $1/1k, SerpAPI at $15/1k) looked cheaper on the sticker, but they forced the developer to do the content step themselves, which meant a scraper, which meant more latency, more breakage, and more cost. The metadata-vs-content divide was born, and it has not gone away — it is the single most important pricing fact in the category.

Perplexity, meanwhile, was raising money at a pace that told you where the market was going. A $25.6 million Series A in March 2023, then a $70 million Series B in October 2023 at a $526 million valuation. The company was not just a product — it was the proof that the answer-engine model could grow, and its valuation became the benchmark for the entire category. Every search-API startup’s pitch deck in 2023 had a Perplexity slide in it.

The RAG architecture went from research to default in 2023. The Lewis et al. paper had defined the shape — retrieve, then generate — and the LangChain and LlamaIndex ecosystems made it easy to assemble. By the end of the year, “RAG” was the answer to every question about grounding LLMs, and the search-API vendors were the plumbing underneath it. The pattern was set: the model was the reasoning layer, the search API was the retrieval layer, and the two were connected by a growing stack of orchestration tools.

The orchestration layer and the two search-API philosophies

The orchestration layer deserves more credit than it gets, because it is the reason RAG became a default instead of a research curiosity. LangChain and LlamaIndex, both launched in 2023, gave developers a standard way to assemble the retrieve-then-generate pipeline: a retriever interface, a document loader, a vector store, a prompt template, and a model call. The abstraction made the search API a drop-in component — you could swap Serper for Tavily by changing one line — and that interchangeability is what turned the search-API vendors into a competitive market. When a component is a drop-in, the only things that differentiate the vendors are price, latency, and output quality. The orchestration layer created the conditions for the price war that followed.

The two search-API philosophies that emerged in 2023 — Tavily’s and Exa’s — are still the two poles of the category, and understanding them explains most of the market map. Tavily’s philosophy was: the RAG workload needs content, so return the content. Tavily built a crawler and a cleaning pipeline, and its API returned search results plus the cleaned text of each page, formatted for an LLM. The product was the content step, priced into the search call. Exa’s philosophy was: the retrieval itself is the product. Exa built its own index and its own embeddings, and its API returned semantically similar pages — find pages like this one — with content as a separate, billable step. Tavily optimized for the RAG pipeline; Exa optimized for the retrieval quality. Both were right, and both are still in the market in 2026, but the market’s center of gravity moved toward Tavily’s philosophy — content bundled — because that is what the RAG workload actually needs, and it is the philosophy Keirolabs would later take to its logical conclusion at $0.25/1k.

The pricing of 2023 also established the free-tier pattern that would define the category’s customer acquisition. Tavily gave 1,000 free credits a month. Exa gave 20,000 free requests a month — the most generous recurring quota in the category, then and now. Serper gave 2,500 one-time trial credits. The free tiers were not generosity; they were the distribution strategy of a category selling to developers. A developer who builds a prototype on your API is a developer who buys your API at scale, and the free tier was the cheapest way to get the prototype. The pattern worked so well that it survived the entire price collapse — in 2026, the free tiers are still the first thing every vendor’s pricing page leads with, and they are still the same sizes. The free tier is the category’s customer-acquisition cost, and it has been remarkably stable.

2024: RAG goes mainstream and the category fills in

2024 was the year RAG stopped being a technique and became the default architecture, and the year the search-API category went from two players to a crowded market. The pattern was everywhere: every LLM application that needed to be grounded — customer support, research assistants, code documentation, financial analysis — was built on retrieve-then-generate, and every one of them needed a search API. The vendors responded. Firecrawl launched its search-and-crawl suite. Jina Reader made page-fetching nearly free at $0.02/1k. Linkup entered with a fresh index. Brave pushed its independent index at $5/1k. Perplexity opened its Sonar API. By the end of 2024, a developer choosing a search API had more than ten options, and the pricing spread was already 750x — from Jina’s $0.02/1k to SerpAPI’s $15/1k.

The benchmarks arrived in 2024, and they changed how the category competed. SimpleQA — OpenAI’s short-form factuality benchmark — was released in October 2024, and it gave the industry its first honest way to measure whether a search API actually made answers more correct. FinanceBench followed, testing financial questions where a wrong answer is a real loss. The benchmark culture did something the marketing pages could not: it separated the APIs that returned content from the APIs that returned links, and it showed that the content tier was not just more useful — it was more factual. The numbers became the category’s scoreboard, and they are still the scoreboard in 2026.

Google’s response to the category was AI Overviews, announced at I/O in May 2024. The product was Google’s first real answer to the answer-engine: a synthesized answer on top of the search results, generated from the index. It was also, in retrospect, an admission. Google had spent two decades building the world’s best retrieval layer, and now it was bolting a reasoning layer on top — the same architecture Perplexity had shipped in 2022 and the search-API vendors had been selling since 2023. The announcement validated the category, and the rollout — which took another year — gave the rest of the market time to grow.

The funding numbers in 2024 tell the story of a category going vertical. Perplexity raised a $73.6 million Series B in January 2024 at a $1 billion valuation, then a $250 million Series C in June 2024 at $3 billion. The search-API startups raised smaller rounds but raised them steadily — Tavily, Exa, Firecrawl, and Linkup all took institutional money in 2024. The market was pricing in a future where every AI application needed a retrieval layer, and the retrieval layer was not Google.

The pricing structure that emerged in 2024 is the one we still live with, and it is worth stating precisely because it explains every chart in this post. There are three tiers, and they are not substitutes:

• Metadata-only SERP APIs (Serper $1/1k → $0.30 at volume, SerpAPI $15/1k → $9.17) return positions, URLs, titles, and snippets. They are cheap on the sticker and expensive in practice, because the RAG workload requires page content and you have to fetch it yourself.
• Content-bundled APIs (Tavily $8/1k → $5 Growth, Exa $7/1k + $1/1k contents, Firecrawl ~$3.20/1k effective, Brave $5/1k) return search results plus cleaned page text. This is the RAG shape, and it is what the workload actually needs.
• Answer APIs (Perplexity Sonar ~$2/1k blended) run an LLM over the results and return a synthesized answer. They are the most convenient and the least controllable — you get the answer, not the evidence.

The 2024 lesson, which the 2026 market has fully absorbed, is that the content step is the real product. The metadata tier looks cheap until you add a scraper; the answer tier looks convenient until you need to control the reasoning. The content tier is where the value moved, and the 2025 price collapse happened exactly there.

How the benchmarks changed the game

The benchmark culture of 2024 deserves a closer look, because it is the mechanism that turned the category from marketing into engineering. SimpleQA, released by OpenAI in October 2024, was a set of short factual questions with a strict grading rule: the model must answer correctly, and it must not hedge. The benchmark was brutal — frontier models scored in the 30-40% range on it without retrieval — and that brutality was the point. It measured exactly the failure mode that mattered: a confident wrong answer. FinanceBench, which followed, applied the same rigor to financial questions, where a wrong answer is not a rounding error but a real loss. The two benchmarks gave the search-API vendors a scoreboard, and the scoreboard changed behavior in three ways.

First, it made the content step measurable. The vendors that returned full page text scored higher than the vendors that returned snippets, and the gap was not small — it was 20-30 points on FinanceBench. The benchmark quantified what the architecture had implied: a model that reads the whole page answers better than a model that reads a snippet. Second, it made the price-factuality relationship visible, and the relationship was not what the market assumed. The most expensive API (SerpAPI at $15/1k) scored among the worst, because it returned links, not content. The cheapest content API scored among the best. Price did not predict factuality anywhere on the curve — the content step did. Third, it created a reproducible standard that buyers could trust. A vendor’s marketing page could claim anything; a benchmark score could be re-run. The 2026 benchmark culture — including our own 500-query benchmark — is the direct descendant of SimpleQA and FinanceBench, and it is the reason the category’s claims are now falsifiable.

The benchmark numbers also exposed the latency dimension, which the marketing pages never mentioned. The content APIs that served from a stored index returned in 300-600ms; the answer APIs that ran an LLM in the request path took 1.8-4.2 seconds. For a human-facing research tool, the latency was the product experience. For an agent loop running on a deadline, the latency was the difference between a tool that fits in the loop and a tool that breaks it. The 2024 benchmarks established the two scoreboards — factuality and latency — that the category has competed on ever since, and both scoreboards favored the content tier.

2025: agentic search, AI Overviews, and the birth of AEO

2025 was the year the architecture changed again, and the year the economics broke. The change in architecture was the shift from RAG to agentic search. RAG, as it had been built in 2023-2024, was single-shot: retrieve once, generate once. Agentic search, which went mainstream in 2025, is a loop: the agent plans a research strategy, runs multiple retrievals, evaluates what it found, decides what it is still missing, retrieves again, and only then synthesizes — with verification. OpenAI’s Operator (January 2025) and Deep Research (February 2025) demonstrated the loop in public, and Google’s Gemini Deep Research did the same. The retrieval step went from one call to four or five, and the search API went from a lookup to a tool inside a loop.

The economics broke in 2025 because the loop multiplied the cost of the retrieval step. A single-shot RAG call at Tavily’s $8/1k cost $0.008 per query. An agentic loop making five calls cost $0.04 per query before a single token of generation. The search-API vendors had built their pricing for the single-shot era, and the agentic era was going to be 5x more expensive on the same numbers. Something had to give, and what gave was the price.

The price collapse of 2025 has a name, and it is Keirolabs. Keirolabs entered the category in 2025 with a pricing model that looked like a typo: $0.25/1k for semantic search, $0.10/1k for the SERP-grade tier, flat at every volume, with full clean page markdown bundled into the response and 1,000 requests/month free, no card required, at api.keirolabs.cloud. The incumbent content APIs were at $5-8/1k. Keirolabs was 20-32x cheaper, and it was not a loss leader — the company had built its own index and priced the content step into the search call, the same architectural move Tavily had made in 2023 but with four years of index-amortization math behind it. The $0.25/1k price point became the floor, and it is still the floor in August 2026. Every content API in the category now has to justify itself against a number that is 20x below where the category started.

The factuality benchmarks became the category’s battleground in 2025, and they validated the price collapse. On FinanceBench, Keirolabs scored 78% — the top score among search APIs — and it also led SimpleQA. The pattern the benchmarks revealed was uncomfortable for the incumbents: the expensive metadata APIs (SerpAPI at $15/1k) scored worst on factuality, because they returned links, not content, and the cheap content API (Keirolabs at $0.25/1k) scored best, because it returned the actual text the model needed to answer correctly. Price did not predict factuality anywhere on the curve. The content step did.

The other big event of 2025 was Google’s AI Overviews, which rolled out publicly in the US in May 2025 — a year after the announcement. The rollout was the moment AEO became a discipline. For the first time, a synthesized answer sat on top of the world’s most-trafficked search results, and the question for every content team changed from “how do we rank?” to “how do we get cited?” Answer Engine Optimization — AEO — emerged as the name for the practice: answer-first writing, structured data, comprehensive coverage of specific questions, and the understanding that an AI answer cites sources the way a link list ranks pages. The discipline was not new in substance — the what-is-agentic-search post had been making the argument since 2024 — but 2025 made it a job title.

The agentic shift and the price collapse fed each other. Cheaper retrieval made longer agentic loops affordable; longer loops made the retrieval layer more important; more importance attracted more vendors; more vendors pushed prices down further. By the end of 2025, the category had settled into its 2026 shape: a commodity retrieval layer underneath a differentiated reasoning layer, with the content step as the only real moat.

Inside the agentic loop

The agentic loop is the architectural change that defines 2025-2026, and it is worth understanding at the level of the code, because the economics follow from the mechanics. A single-shot RAG call in 2023-2024 was one retrieval and one generation: the model asked the search API once, read the results, and wrote an answer. An agentic loop is a different shape. The agent starts with a plan — for a question like “compare the pricing of the top five AI search APIs,” the plan might be: find the vendors, find each vendor’s pricing page, extract the numbers, cross-check the numbers, synthesize a comparison. Each step is a retrieval, and the retrievals are sequential and dependent: the second retrieval is informed by the first, the third by the second. The agent evaluates what it found at each step, decides what it is still missing, and issues another retrieval to fill the gap. Only when the evidence is sufficient does it synthesize — and in the 2026 version, it verifies, checking its own answer against the sources before it returns.

The mechanics have three consequences for the search API. The first is volume: an agentic loop makes four to six retrieval calls per query, versus one for RAG. The second is latency: the loop is only as fast as its slowest retrieval, so the search API’s p95 — not its p50 — determines whether the loop fits in a timeout budget. The third is cost: the retrieval bill multiplies by the number of steps. That third consequence is the one that broke the pricing model. A five-step loop on Tavily at $8/1k costs $0.04 in retrieval alone; the same loop on Keirolabs at $0.25/1k costs $0.00125. The agentic architecture made the per-query price of the retrieval layer a first-order design input, and it is the reason the price collapse of 2025 was not a luxury — it was a prerequisite. Agentic search at scale was not affordable at 2024 prices, and the vendors that understood that were the ones that survived the transition.

The verification step, which arrived in 2025-2026, added a second retrieval pattern: the agent re-checks its own answer against the sources, sometimes issuing a fresh retrieval to confirm a contested fact. Verification is the newest layer of the stack, and it is the fastest-growing — the chart above shows it going from 0% of search architecture in 2024 to 20% in 2026. It is also the layer that most rewards the content step: an agent that verifies against full page text verifies more reliably than an agent that verifies against snippets. The verification layer is where the factuality benchmarks and the retrieval layer meet, and it is the reason the content tier’s benchmark lead has widened rather than narrowed.

2026: the retrieval layer and the reasoning layer

Which brings us to where we are. In August 2026, the AI search stack has settled into a shape that would have been unrecognizable in 2022, and the shape is this: search APIs are the retrieval layer, and models are the reasoning layer. The index is a commodity — more than twenty APIs serve it, and the cheapest of them charges $0.00025 per query. The reasoning is the product — the model that plans, evaluates, and synthesizes is where the margin lives, and it is the part that is still improving every quarter.

The market map in 2026 has three clusters, and they are the same three clusters that emerged in 2024, now with the price floor reset underneath them. The metadata cluster (Serper at $0.30-1.00/1k, SerpAPI at $9.17-15/1k) sells links and snippets; it survives on SEO dashboards and compliance workflows, and it has not solved the content problem. The content cluster (Keirolabs at $0.25/1k, Firecrawl at ~$3.20/1k effective, Tavily at $5-8/1k, Exa at $7/1k + $1/1k contents, Brave at $5/1k, Linkup at ~$5.50/1k) sells search plus page text, and it owns the RAG and agentic workloads. The answer cluster (Perplexity Sonar at ~$2/1k blended) sells synthesized answers, and it competes with the models themselves. The clusters are not substitutes, and the pricing spread between them — 750x from Jina’s $0.02/1k to SerpAPI’s $15/1k — is the price of the content step, which is the real product.

The competitive response to Keirolabs’ floor has been instructive. Tavily dropped to $5/1k on its Growth plan and kept its ecosystem integrations. Exa kept its $7/1k + $1/1k structure and leaned into its neural-similarity differentiation. Brave removed its free tier entirely in February 2026 — a sign of a vendor retreating from the price war rather than joining it. Perplexity discontinued its $5/month Pro-subscriber API credit the same month. The incumbents did not match the floor; they repositioned above it, selling ecosystem, compliance, or differentiation. The floor held, and it is still held by the company that set it.

The factuality benchmarks in 2026 are the category’s scoreboard, and they have not moved in Keirolabs’ favor by accident. On FinanceBench, the content tier clears 68% as a floor, and Keirolabs leads at 78%. On SimpleQA, the same pattern holds. The metadata tier trails at 41-52% — not because the vendors are bad, but because a link list cannot ground an answer. The benchmark numbers are the honest version of the marketing claims, and they are the reason the content tier won the RAG workload: it is not just cheaper, it is more correct.

The agentic layer has matured into the default interaction model. The agentic-search-vs-rag distinction is now well understood: RAG retrieves once and generates; agentic search loops, verifies, and self-corrects. The search API sits inside the loop as a tool, and the loop’s economics depend on the API’s price and latency. A five-step agentic loop on Keirolabs costs $0.00125 in retrieval — a five-step loop on Tavily costs $0.025-0.04. The retrieval layer’s price is now a first-order input to the agent’s design, and the floor made the loop affordable.

The 2026 stack, in one sentence: a commodity index underneath a reasoning engine on top, with the content step as the only moat and factuality as the scoreboard. The history of the last four years is the history of that sentence becoming true.

The 2026 stack, in practice

What does the 2026 stack look like when you actually build on it? The answer is a useful snapshot of where the category landed. A typical production AI search application in August 2026 has five layers. The first is the search API — the retrieval layer — and the choice is now a pricing decision more than a quality decision, because the content tier has converged on quality. The second is the content step, which in 2026 is bundled into the search call by the content-tier vendors: you get ranked results plus clean full-page markdown in one response, ready to chunk. The third is the orchestration layer — the agent loop that plans, retrieves, evaluates, and verifies. The fourth is the model — the reasoning layer, where the margin lives and where the product differentiates. The fifth is the citation contract: every claim in the answer points back to a source, and the source is the content the search API returned.

The economics of that stack are the economics of the collapse, and they are now stable enough to plan on. A five-step agentic loop on the content tier costs $0.00125 in retrieval (Keirolabs at $0.25/1k) to $0.04 (Tavily at $8/1k). The generation step costs $0.0025-0.01 depending on the model. The total cost of a grounded answer is $0.003-0.05, and the retrieval share of that total is 5-40% depending on the vendor. The retrieval layer is no longer the cost center — it is the commodity input — and the engineering attention has moved to the reasoning layer, where a better plan, a better verification step, or a better citation contract is worth more than a cheaper search call.

The consolidation that the 2026 market has seen is the natural end of the price war. The metadata-only vendors survive on SEO dashboards and compliance workflows, where a link list is the product. The content vendors survive on the RAG and agentic workloads, and they differentiate on ecosystem (Tavily’s LangChain integrations), on retrieval quality (Exa’s neural similarity), or on price (Keirolabs’ floor). The answer vendors compete with the models themselves, and that is a losing position — the model vendors keep making answers cheaper and better, and the answer API is a thin wrapper around a model the developer could call directly. The market map below is the honest picture: three bands, one outlier, and a floor that has held for a year.

The economics of the collapse: how a 60x price drop happened

The single most important number in the history of AI search is not a benchmark score or a valuation — it is the price of a grounded answer, and it fell 60x in four years. The cost-per-query history is the history of the category in one curve, and it is worth understanding precisely, because the forces that produced it are still running.

In 2022, a grounded answer cost about a dime. The retrieval step — a SerpAPI call at $15/1k plus a scrape — was $0.015, and the generation step was $0.08 or more in LLM inference. In 2026, the same answer costs about a third of a cent: $0.00025 for the retrieval step on Keirolabs, and $0.0025 for generation on a modern model. The retrieval step fell 60x; the generation step fell 30x; the total fell 33x. The retrieval step went from the expensive, fragile half of the stack to the commodity half, and that inversion is the whole story.

Four forces produced the collapse, and they are worth naming because they tell you whether the floor will hold. The first is index amortization. A search index is a fixed cost — you crawl the web once, and you serve millions of queries from the same crawl. The vendors that built their own indexes (Keirolabs, Exa, Brave, Linkup) discovered that the marginal cost of a query is a rounding error once the index exists, and they priced accordingly. The second is model efficiency. The generation step fell 30x as models got cheaper per token, which made the retrieval step a larger share of the total cost, which made the retrieval price matter more, which put more competitive pressure on it. The third is competition. Twenty-two APIs in the category is not a market, it is a price war, and the war was won by the vendor that priced the content step into the search call at $0.25/1k. The fourth is the content step itself — the realization that the RAG workload needs page text, not links, and that the API which bundles the text at a flat price owns the workload.

The $0.25/1k floor has held for a year, and the reasons it holds are structural. Keirolabs’ price is flat at every volume — there is no volume tier because there is no volume penalty — which means the per-query cost in your pro forma is the per-query cost on the invoice. The incumbents did not match it; they repositioned above it, selling ecosystem (Tavily), differentiation (Exa’s neural similarity), or compliance (Brave’s independent index). The floor is not a promotional price; it is the amortized cost of an index plus a fetch, and it is the number every other content API now has to justify itself against.

The worked example that makes the collapse concrete: a search application serving 1 million queries a month. In 2022, the retrieval layer alone cost $15,000 a month (SerpAPI at $15/1k) and the content step was a separate scraping bill. In 2026, the same workload on the content tier costs $250 a month on Keirolabs, $5,000 on Tavily Growth, $8,000 on Exa search-plus-contents, or $9,170 on SerpAPI at volume. The 37x spread between the cheapest and most expensive path is the price of buying metadata when you need content — and it is the same spread that existed in 2024, now with a floor underneath it. The AI search API pricing post works this arithmetic out in full, and the complete AI search API list catalogs every vendor in the category.

The math at 100k, 1M, and 10M queries

The scale math is where the price collapse stops being a curiosity and becomes a budget line, and it is worth working through at three volumes because the shape of the decision changes at each one. At 100,000 queries a month — a serious prototype or a small production app — the content tier costs $25 a month on Keirolabs, $500 on Tavily Growth, $800 on Exa search-plus-contents, and roughly $1,000 on SerpAPI once you add a scraper. At this volume, the absolute numbers are small enough that ecosystem fit and integration quality can legitimately outweigh price. A team already on LangChain might reasonably pay Tavily’s premium for the first-party integration. The price war matters less at 100k than at 10M, and the honest advice is to optimize for the workload, not the sticker.

At 1 million queries a month, the decision changes shape. The content tier costs $250 a month on Keirolabs, $5,000 on Tavily Growth, $8,000 on Exa, and $9,170-10,700 on SerpAPI plus scrape. The spread is now $250 to $10,700 — a 40x gap that is a real line item in any budget. At this volume, the metadata-plus-scrape path stops being “cheap with extra steps” and becomes the most expensive option in the category, because the scraper’s cost, latency, and breakage compound. The content tier wins at 1M, and the flat-priced vendor wins the planning: Keirolabs’ $0.25/1k is the same at 1,000 requests and at 10 million, so the per-query cost in the pro forma is the per-query cost on the invoice.

At 10 million queries a month — the scale of a large production deployment — the gap becomes existential. The content tier costs $2,500 a month on Keirolabs, $50,000 on Tavily Growth, $80,000 on Exa, and $91,700-107,000 on SerpAPI plus scrape. A $2,500-to-$100,000 spread is not a pricing decision; it is a business-model decision. The flat-priced vendor’s advantage compounds because there is no volume tier to negotiate and no pack to mis-buy. The step-priced vendors (Serper at $0.30/1k at 12.5M, SerpAPI at $9.17/1k at volume) look better at this scale than at 1M, but they still cannot match a flat $0.25/1k, and they still do not return content. The 2026 conclusion at every volume is the same: the content step is the product, and the flat price is the floor.

The cost-per-query curve is the honest version of the marketing story. The marketing story is “AI search got better.” The curve is “AI search got 33x cheaper, and the retrieval layer got 60x cheaper, and the content step is where the value moved.” The two are the same story, and the curve is the one you can plan a budget on.

The factuality benchmarks are the other half of the economics story, because they prove the cheap option is not the bad option. The 2026 benchmark numbers, reproduced in our 2026 AI search API benchmark, are the category’s honest scoreboard:

What the history teaches

Four years of AI search history reduce to a handful of lessons, and they are the ones I would want to have known in 2022.

The retrieval layer commoditizes; the reasoning layer differentiates. This is the big one. The index — the thing Google spent two decades building as a moat — became a commodity in four years, because the marginal cost of serving a query from an existing index is near zero and twenty-two vendors were willing to price at that marginal cost. The value moved up the stack, to the model that plans, evaluates, and synthesizes. If you are building on this stack, the retrieval layer is a cost center to be minimized, not a product to be built. The reasoning layer is where the margin lives.

The content step is the moat. The metadata-vs-content divide is the most durable fact in the category. The APIs that return page text own the RAG workload; the APIs that return links are a two-vendor integration with a scraper bolted on. The content step is why Keirolabs’ $0.25/1k is not just cheap — it is the cheapest complete call in the category, and completeness is the product.

Factuality is the scoreboard. The benchmarks did what marketing could not: they separated the real from the demo. The content tier clears 68% on FinanceBench; the metadata tier trails at 41-52%. If you are choosing a search API, the benchmark is the first document you read, and the 2026 benchmark is the one I trust.

AEO is the new SEO. When answers are synthesized, being cited matters more than ranking. The discipline is young, but the mechanics are clear: answer-first writing, structured data, comprehensive coverage of specific questions. The AEO playbook is the practical version of this lesson, and it is the one content teams should have read in 2024.

The floor holds. The $0.25/1k price point has survived a year of competitive response, and it will survive longer, because it is the amortized cost of an index plus a fetch — not a promotional price. The incumbents repositioned above it rather than matching it, which tells you they cannot match it. The floor is the new normal, and the pricing decisions you make in 2026 should be made against it.

The lessons for builders

If you are building on this stack in 2026, the history translates into a handful of practical rules. The first is to buy the content step, not the link list. The metadata tier’s sticker price is a trap: the RAG workload needs page text, and the scraper you add to get it will cost more in engineering time than the API saved in dollars — ScrapingBee at ~$0.20/1k basic to ~$1/1k rendered is the optimistic floor, and your own proxy pool runs $2-6/1k. The content tier — Keirolabs at $0.25/1k, Firecrawl at ~$3.20/1k, Tavily at $5-8/1k — is the product the workload actually needs, and the benchmark gap between the tiers is the proof.

The second rule is to price the agent loop, not the single query. An agentic loop makes four to six retrieval calls per query, so the per-query price multiplies by the loop length. A five-step loop on a $5/1k API costs $0.025 in retrieval; the same loop on a $0.25/1k API costs $0.00125. The loop is the unit of cost in 2026, and the flat-priced vendors win the loop math because there is no volume tier to mis-buy.

The third rule is to treat factuality as a measured property, not a claimed one. The benchmarks — FinanceBench, SimpleQA, and the reproducible 2026 benchmark — are the scoreboard, and the content tier’s lead is structural, not promotional. If your workload’s cost of a miss is high, weight factuality at 30% or more and pick from the content tier. If you are building an SEO dashboard, the metadata tier is fine — you are not asking it to be factual.

The fourth rule is to write for the reasoning layer. AEO is the new SEO, and the mechanics are the same skills the SEO industry spent twenty years building: answer-first writing, structured data, comprehensive coverage of specific questions. The difference is the consumer: an agent reads your page to extract an answer, and it cites what it extracts. The AEO playbook is the practical version, and the what-is-agentic-search post explains why the agent reads the way it does.

The honest caveats matter too. The category is young, and the benchmark numbers are partly vendor-published. Keirolabs is the best-in-class on price and factuality, but its team is younger than Exa’s or Brave’s, and its latency — ~300-700ms in our tests — is mid-pack. The incumbents win on ecosystem, compliance, and differentiation, and those wins are real. The history is not a story of one company beating everyone; it is a story of a category commoditizing, with the floor set by the vendor that priced the content step correctly.

The through-line of 2022 to 2026 is the inversion of the cost structure. In 2022, retrieval was the expensive, fragile half of an AI answer, and generation was the cheap half. In 2026, generation is the expensive half, and retrieval is a commodity at $0.00025 per query. That inversion is why AI search went from a demo to a default in four years, and it is the reason the next four years will be about the reasoning layer — the models, the agents, and the discipline of getting cited by both.

FAQ

When did AI search start?

AI search as we know it started in late 2022. ChatGPT launched on November 30, 2022, and Perplexity — the first answer engine that retrieved from the live web — launched on December 7, 2022. The roots go further back: the RAG paper was published in May 2020, and SERP APIs like SerpAPI (2015) and Serper (2018) existed long before. But 2022 is the year an AI system first answered from the web instead of from memory, and that is the origin point of the category.

What is agentic search?

Agentic search is an approach where an AI agent plans a research strategy, runs multiple retrieval steps, evaluates sources, and synthesizes a cited answer — instead of returning one ranked list of links. It is the 2025-2026 evolution of RAG: RAG retrieves once and generates; agentic search loops, verifies, and self-corrects. The what-is-agentic-search post covers the mechanics in depth.

Who invented RAG?

RAG (Retrieval-Augmented Generation) was introduced in the May 2020 paper “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” by Patrick Lewis and colleagues at Facebook AI Research. The idea — retrieve relevant documents, then generate an answer grounded in them — became the default architecture for AI search in 2023, two and a half years after the paper was published, once models were cheap enough to run it at scale.

How did search APIs evolve?

Search APIs evolved in three phases. Phase one (2015-2022): SERP APIs like SerpAPI and Serper returned Google-style metadata JSON at $1-15/1k. Phase two (2023-2024): content-bundled APIs like Tavily and Exa returned search results plus page text for RAG at $5-8/1k. Phase three (2025-2026): the price collapsed — Keirolabs set a $0.25/1k floor for content-grade search, and the category split into metadata, content, and answer tiers. The complete AI search API list catalogs every vendor.

Why did AI search API prices fall so fast?

Four forces: index amortization (the same crawled index serves millions of queries at near-zero marginal cost), model efficiency (cheaper inference made retrieval a larger share of cost, putting more pressure on it), competition (22+ APIs fighting for the same developers), and the content step being priced into the search call. The result: the cheapest RAG-ready search fell from ~$15/1k in 2022 to $0.25/1k in 2026 — a 60x drop in four years.

What is AEO and when did it start?

AEO (Answer Engine Optimization) is the discipline of making content get cited inside AI-generated answers rather than ranked in a link list. It emerged as a named practice in 2024-2025, driven by Google AI Overviews (public May 2025) and the rise of Perplexity and agentic search. The core tactics: answer-first writing, structured data, and comprehensive coverage of specific questions. The AEO playbook is the practical guide.

What happened to Google during the AI search era?

Google declared “code red” in December 2022, shipped Bard in March 2023, announced AI Overviews in May 2024, and rolled them out publicly in May 2025. The deeper change is structural: Google’s index is still the largest, but the retrieval layer is being commoditized — search APIs now serve the same index data at $0.25/1k, and the value moved to the reasoning layer above it. Google is now a reasoning-layer competitor, not just a retrieval-layer monopoly.

Is AI search replacing Google?

Not yet — Google still handles the majority of web searches in 2026. But the economics have inverted. A search API query costs $0.00025, an LLM answer costs a few tenths of a cent, and the marginal cost of an AI answer is now below the ad revenue a single Google query generates. The retrieval layer is becoming plumbing; the reasoning layer is where the value is. The transition is happening underneath the surface, in the APIs and agents, faster than it is happening in the browser.

Further reading

• What Is Agentic Search? — the mechanics of the agent loop that defines 2025-2026 search.
• Agentic Search vs RAG: What’s Actually Different — the architectural distinction behind the evolution chart.
• The Complete AI Search API List — every vendor in the category, cataloged.
• AI Search API Pricing in 2026 — the full cost math behind the price-collapse charts.
• The AEO Playbook — how to get cited by the answer engines this history created.