Google’s Jeff Dean: AI Search relies on classic ranking and retrieval
In the rapidly evolving landscape of artificial intelligence, there is a common misconception that the advent of Large Language Models (LLMs) has completely rewritten the rules of information retrieval. Many observers assume that Google’s transition toward AI-driven results, such as AI Overviews, represents a total abandonment of the “old” search algorithms that have governed the web for decades. However, according to Jeff Dean, Google’s Chief AI Scientist, the reality is far more grounded in tradition than many realize. In a detailed interview on the Latent Space: The AI Engineer Podcast, Dean pulled back the curtain on the architecture powering Google’s modern AI search experiences. His insights reveal a critical truth for developers, SEO professionals, and tech enthusiasts: AI search is not a replacement for classic search infrastructure. Instead, it is a sophisticated layer that sits on top of a foundational system built on decades of ranking, retrieval, and indexing expertise. The Architecture: Filter First, Reason Last The core of Jeff Dean’s explanation centers on a concept that might surprise those who view AI as an all-knowing entity that “reads” the entire internet in real-time. He clarified that Google’s AI systems do not process the whole web simultaneously for every query. Instead, they follow a rigorous, multi-stage pipeline designed for efficiency and accuracy. Dean describes this as a “staged pipeline” that prioritizes filtering before any generative reasoning occurs. Visibility in an AI-generated search result still depends entirely on a document’s ability to clear traditional ranking thresholds. If a piece of content does not make it into the broad candidate pool of search results through standard SEO and ranking signals, it has zero chance of being used by an LLM to synthesize an answer. In essence, the AI doesn’t find the content; the search engine finds the content, and the AI merely explains it. The Candidate Pool: From Trillions to Thousands To understand how this works at scale, we must look at the numbers Dean provided. The internet consists of trillions of tokens—fragments of data that make up the web. When a user enters a query, it is computationally impossible and wildly inefficient for a high-reasoning LLM to scan those trillions of tokens to find an answer. Instead, Google uses “lightweight methods”—the classic retrieval systems—to narrow the field. This first pass identifies a subset of roughly 30,000 documents that are potentially relevant to the user’s intent. This initial culling is done in milliseconds using traditional signals. Dean explained that this process is about “down-ranking” the noise to find a manageable set of “interesting tokens.” Reranking and Refining Once the system has identified the top 30,000 candidates, it doesn’t stop there. Google applies increasingly sophisticated algorithms and signals to refine that list further. This is a tiered process where the cost of computation increases as the number of documents decreases. The system filters the 30,000 documents down to a few hundred, and eventually down to the final set—often around 10 to 100 documents—that are truly relevant to the specific task. Dean refers to the user experience of AI search as an “illusion” of attending to the entire web. While it feels like the AI is searching the whole internet for you, it is actually only “paying attention” to the very small subset of data that the traditional ranking engine has already verified as high-quality and relevant. “You’re going to want to identify what are the 30,000-ish documents… and then how do you go from that into what are the 117 documents I really should be paying attention to?” Dean noted. Matching Intent: Moving from Keywords to Meaning One of the most significant shifts in search over the last several years has been the move from lexical matching (finding exact words) to semantic matching (understanding the meaning behind words). While LLMs have accelerated this trend, Dean pointed out that this evolution is not entirely new; it is a continuation of a journey Google started long ago. In the early days of search, if a user typed “blue suede shoes,” the engine looked for pages that contained those exact three words. If a page used the phrase “azure leather footwear,” it might not show up, even though it was contextually identical. Today, thanks to LLM-based representations of text, Google can move beyond “hard” word overlap. The Power of Topic Overlap Dean explained that LLMs allow Google to evaluate whether a page—or even a specific paragraph within a page—is topically relevant to a query, even if the wording differs entirely. This shift places a premium on topical authority and comprehensive coverage. For content creators, this means that repeating a keyword five times is far less effective than explaining a concept so clearly that the system understands the subject matter’s intent. This “softening” of the definition of a query allows Google to bridge the gap between how people think and how they type. By using LLM representations, the search engine can map the “meaning” of a query to the “meaning” of a document, creating a much more fluid and intuitive discovery process. The 2001 Milestone: Why Query Expansion Changed Everything To provide context for today’s AI advancements, Jeff Dean took a trip down memory lane to 2001. This was a pivotal year for Google, marking the moment when the company moved its entire index from physical disks into RAM (memory) across a massive fleet of machines. Before 2001, adding extra terms to a user’s query was expensive. Every time Google wanted to look for a synonym, it required a “disk seek,” which added latency and slowed down the search for the user. Consequently, the engine had to be very selective about the terms it searched for. Query Expansion in the Pre-LLM Era Once the index was in memory, the technical constraints vanished. Google could suddenly take a three-word query from a user and “expand” it into 50 terms behind the scenes. If a user searched for “cafe,” the system could simultaneously look for “restaurant,” “bistro,” “coffee shop,” and “diner” without any performance penalty. Dean emphasized that