Author name: aftabkhannewemail@gmail.com

Not long ago, retrieval-augmented generation (RAG) was hailed as the definitive future of digital search. Early conversations around Google’s Search Generative Experience (SGE)—which has since matured into AI Overviews—framed RAG as a modern marvel designed to solve the limitations of large language models. The architecture was simple: a user query went in, a retriever fetched the top matching document chunks, an LLM read those chunks, and a synthesized answer with inline citations was served to the user. That linear, single-shot pipeline is now obsolete. Every major search engine and AI platform has quietly transitioned to a highly sophisticated, multi-layered framework. If you look at Google AI Mode, ChatGPT Search, Perplexity Pro Search, Gemini Deep Research, or Microsoft Copilot, they no longer rely on a simple retrieve-and-generate mechanic. Instead, they execute dynamic plans, switch fluidly between distinct tools, perform multi-hop retrievals, self-correct, and grade their own intermediate work. This is the era of agentic RAG, and it has fundamentally rewritten the rules of Generative Engine Optimization (GEO). If your optimization strategies are still designed to rank inside a single, static retrieval window, you are optimizing for systems that no longer exist. To survive this change, you must understand how agentic search works, how major search engines are building it, and how to adapt your content architecture to win at every stage of the agentic loop. What Traditional RAG Got Right—and What Has Changed The core thesis of the early RAG era remains true: passage-level retrieval is still the fundamental unit of relevance in modern search. Static information retrieval (IR) scores no longer dictate search success. Modern systems exist primarily to minimize Delphic costs—the cognitive and temporal cost a user incurs to find and synthesize a definitive answer. Historically, search engines treated organic traffic as a necessary bridge; agentic search engines treat that same traffic as an inefficiency they must solve by delivering complete answers directly to the user. While those principles hold steady, the architecture of the retrieval pipeline has shifted entirely. In 2023, RAG acted like a factory assembly line. The query was converted into dense vector embeddings, a vector database returned the top-k most similar passages, and those passages were fed directly into the LLM’s context window. Sourcing was straightforward because the retrieval set was identical to the citation set. Today, the retrieval pipeline is non-linear and dynamic. It is defined by four core capabilities: planning, tool selection, multi-hop iteration, and self-reflection. Instead of a single retrieval event, a single user prompt now triggers an orchestration loop that can execute five, ten, or twenty sub-retrievals. The search agent evaluates each piece of returned evidence, decides if it needs more context, and only builds the final response when its criteria are fully met. Why Naive RAG Broke Down Naive, single-pass RAG systems inevitably hit a hard ceiling when faced with real-world complexity. Standard vector-similarity search was plagued by four distinct failure modes that made it unsuitable for production-grade search engines: Inability to handle compound queries: A highly specific search like “How does a 1031 exchange interact with a SEP IRA for an LLC owner under 50?” requires multiple distinct lookups. A single vector search can match articles about 1031 exchanges or articles about SEP IRAs, but it cannot bridge the two. The LLM is forced to hallucinate a connection because it was never allowed to retrieve the underlying documents for both concepts independently. No recovery from poor initial retrievals: If the retriever pulls incorrect, stale, or poorly chunked documents during its single pass, the LLM has no safety net. Lacking any mechanism to realize it has bad data, it generates an answer based on faulty context, triggering hallucinations. Zero routing between diverse tools: Not every search question is best answered by a semantic vector search. Live stock prices, mortgage rates, or local weather require API integrations. Complex tax calculations require a code interpreter. Authority-driven lookups require precise lexical keyword filters. Classic RAG systems could not intelligently route queries to the correct technical utility. No self-grading or editorial oversight: Traditional RAG models generate an answer and immediately output it to the user. There is no feedback loop, no sanity check, and no validation process to determine if the synthesized output contradicts its own referenced sources. To solve these critical failure modes, AI engineers integrated reasoning loops and agentic workflows directly into the retrieval framework, turning RAG into a stateful, iterative conversation. Decoding the Four Pillars of “Agentic” RAG To understand agentic RAG, we must move past marketing buzzwords and look at its precise structural definitions. A retrieval architecture is truly agentic only when it exhibits four operational properties: 1. Dynamic Planning Before executing any search, the system acts as a planner. It analyzes the user’s intent and decomposes a complex prompt into an execution plan containing multiple sub-queries. The conceptual model for this process stems from the ReAct framework (Yao et al., 2022), which demonstrated that combining reasoning traces with task-specific actions allows LLMs to iteratively update and execute plans while interacting with external environments or databases. 2. Tool Use and Function Calling Search is no longer a monolith; it is an array of tools. The agent acts as a router that evaluates each sub-query and decides which tool is best suited to retrieve the answer. It can query vector databases, execute structured SQL statements, trigger API endpoints, run a local Python script inside a code interpreter, or crawl live URLs. This behavior is built on the foundation of Toolformer (Schick et al., 2023), proving that language models can autonomously decide when, how, and with what parameters to call external APIs to ground their predictions. 3. Multi-Hop Iteration An agent does not retrieve once and stop. It retrieves, parses the results, identifies missing entities or logical gaps, and uses those new insights to formulate a second or third round of targeted queries. As outlined in the IRCoT (Iterative Retrieval-Cognitive Thoughts) paper (Trivedi et al., 2022), interleaving chain-of-thought generation with multi-step retrieval loops dramatically improves factual accuracy in complex question-answering tasks.

Why your B2B PPC metrics may be lying to you The modern B2B marketing landscape offers advertisers more sophistication and data granularity than ever before. In the early days of search engine marketing, evaluating the success of paid search (PPC) was straightforward, if incomplete. Advertisers relied almost exclusively on basic, surface-level conversions, such as direct form fills on a landing page or simple contact requests. Today, the major ad platforms have evolved. By integrating CRM systems and advanced tracking setups, B2B advertisers can feed a massive volume of deep funnel, offline conversion tracking data back into Google Ads and Microsoft Ads. This data pipe allows systems to optimize bidding strategy not just for initial clicks, but for real business progression. However, this abundance of data introduces a new strategic risk: the urge to measure and optimize for every single metric. When you try to make every micro-action a key performance indicator (KPI) and direct your bidding algorithms to maximize everything at once, you run the risk of succeeding at nothing. The numbers in your ad dashboard may show spectacular growth, while your actual sales pipeline remains completely flat. To understand why your B2B PPC metrics might be lying to you, we must examine how conversion actions are structured, how automated bidding systems digest data, and how to measure true incremental business value. The Illusion of Growth: How Tracking Everything Dilutes Performance It is common for B2B search marketers to implement offline conversion tracking and immediately notice a massive spike in total conversions. On paper, the campaign appears to be performing better than ever. Yet, when the marketing team meets with the sales department, the feedback is discouraging: there is no corresponding increase in actual closed-won revenue or qualified pipeline. Why does this discrepancy happen? The issue usually stems from conversion configuration. When setting up offline conversions, advertisers frequently add multiple stages of the buyer journey—such as raw leads, marketing qualified leads (MQLs), sales qualified leads (SQLs), and sales opportunities—and set them all to primary conversion actions. By marking every stage as a primary conversion action, the advertising platforms treat each step as an independent, valuable event. If a single user clicks an ad, downloads a whitepaper, fills out a contact form, passes the criteria to become an MQL, and is subsequently accepted as an SQL, the system may record four separate conversion events. In reality, you have acquired exactly one prospective customer. This duplicate and quadruple-counting of the buyer journey inflates your conversion volume and artificially lowers your reported Cost Per Acquisition (CPA). This structural flaw also distorts your platform-reported Return on Ad Spend (ROAS). If you have assigned conversion values to each of these actions—a practice that is highly recommended when managed correctly—the platform will aggregate these values. The math becomes circular and deceptive. You see a rising ROAS curve in your Google Ads dashboard that is completely disconnected from real-world bank deposits. Furthermore, evaluating performance solely on average CPA can mask systemic inefficiencies. Average CPA is a aggregate metric that hides your marginal CPA—the actual cost associated with acquiring one additional conversion as your media spend scales. As you push your PPC budgets higher, the cost to capture the next incremental customer often rises sharply. Without analyzing these marginal costs, you may find yourself overpaying dramatically for late-stage conversions at the high end of your budget scale. Establishing a Balanced Conversion Valuation Framework Assigning monetary values to non-transactional B2B actions is highly beneficial, yet many B2B organizations hesitate to implement it. The most common objection is that the true value of a conversion is unknown at the moment the lead is generated. In a complex B2B sales cycle, a lead can take six months or more to progress to a closed deal, and the eventual contract value can vary from thousands to millions of dollars. While utilizing precise, closed-won CRM values is the ultimate goal, you do not need perfect data to start. Instead, you can establish relative, arbitrary values that reflect the progression of your conversion funnel. This model guides the automated bidding algorithms by signaling which actions are most desirable. Consider a relative valuation framework structured on a 10x progression model: Video View: Value of $1 Ungated Asset Download: Value of $10 (10x a video view) Form Fill / Lead Capture: Value of $100 (10x an asset download) Marketing Qualified Lead (MQL): Value of $1,000 (10x a form fill) In this framework, the MQL is sourced via offline conversion data, while the top-of-funnel actions are tracked directly via on-site tags. By valuing an MQL 1,000 times higher than a video view, you instruct the bidding algorithm that you would far prefer a single qualified prospect over 999 casual video views. This prevents the system from taking the path of least resistance—which is often optimizing for the easiest, cheapest, and lowest-intent actions. Once you implement relative values, it is critical to continually validate them against real-world performance. If your relative values are set too high for lower-funnel actions, or conversely, if the gap between a soft lead and a qualified opportunity is too narrow, the algorithm may default to chasing high volumes of cheap, low-quality form fills. A recent real-world scenario illustrates this dynamic. A B2B client was generating a high volume of raw leads, but their MQL and SQL conversion rates were critically low. The account was optimizing for both raw leads and MQLs, but because the value gap was too narrow, the automated bidding system focused its delivery on the easier-to-get raw leads. By reducing the conversion value assigned to raw leads by a factor of 10, the value of MQLs and SQLs became significantly higher in relative terms. This shift altered the signals sent to the ad platform’s bidding algorithm. Within two weeks of implementing this adjustment, the volume of MQLs and SQLs increased significantly, while raw lead volume stayed flat. Although overall lead volume did not grow, the quality of those leads improved, resulting in a more efficient use of the ad

The Shift from Blue Links to Conversational Answers The landscape of search is undergoing its most profound transformation since the invention of the search engine. For years, local search engine optimization (SEO) was a predictable game of ranking in the “Local Pack” or “Map Pack” on Google, building citations, and maintaining consistent Name, Address, and Phone number (NAP) data. While those elements remain foundational, the emergence of artificial intelligence has introduced a new frontier: Generative Engine Optimization (GEO) and AI visibility. Today, users are increasingly turning to AI-driven search experiences. Platforms like Google Gemini (formerly Bard), OpenAI’s SearchGPT, Perplexity AI, and Apple Intelligence are changing how consumers find local services. Instead of typing fragmented keywords like “plumber near me,” users are asking complex, conversational questions: “I have a leaking copper pipe in my basement and need a highly-rated plumber in North Portland who can come out tonight. Who should I call?” For agencies and local business owners, the goal is no longer just ranking in traditional search engine results pages (SERPs). The new challenge is ensuring your clients’ businesses are the ones synthesized, cited, and recommended by AI models. This guide breaks down the strategic blueprint for achieving high AI visibility and securing coveted spots in AI search results. How AI Search Engines Process Local Queries To optimize for AI visibility, it is essential to understand how large language models (LLMs) and generative search engines retrieve and present local information. Unlike traditional search engines that rely heavily on crawling links and indexing keywords, AI engines use a process called Retrieval-Augmented Generation (RAG). When a user asks an AI engine for a local recommendation, the system performs a multi-step process: Query Understanding: The AI analyzes the intent, location, constraints (e.g., “open now,” “pet-friendly,” “wheelchair accessible”), and sentiment of the user’s prompt. Information Retrieval: The AI queries a variety of high-authority databases, web indexes, review sites, and structured data sources to gather potential candidates. Synthesis and Ranking: The model evaluates the options based on proximity, authority, specific match to the user’s constraints, and online reputation. Response Generation: The AI writes a natural-sounding response, often listing two or three top recommendations complete with justifications, and links back to the source material as citations. If a local business does not have a robust, clear, and highly authoritative digital footprint across the platforms these AI engines crawl, it simply will not exist in the generative output. Keyword Research Reimagined for AI Visibility Traditional keyword research focuses on search volume and keyword difficulty. In the age of AI, however, keyword research must evolve to focus on natural language patterns, intent, and contextual queries. AI models excel at understanding context, which means optimizations must be more semantic and descriptive. From Keywords to Entities In modern SEO, search engines view the world in terms of “entities” (real-world things, places, people, and concepts) rather than mere text strings. A local business is an entity. To get an AI to recommend your client, you must build strong semantic relationships between your client’s business entity and the specific attributes, services, and locations they cover. Instead of optimizing solely for “dentist in Chicago,” you must optimize for the entity relations: [Dentist Name] offers [Invisalign] in [Lincoln Park, Chicago] and has [wheelchair-accessible facilities] with [free parking]. AI engines crawl the web to build these relational maps. The more consistently these connections are stated across the web, the more confident the AI will be in recommending the business. Targeting Long-Tail, Conversational Queries To align with how users speak to AI chatbots, perform keyword research that uncovers long-tail, conversational queries. Use tools like AnswerThePublic, Google’s “People Also Ask” feature, and mining customer service emails or chat logs to find specific questions. Focus on: Problem-solving queries: “How do I fix a drafty window in an old house?” Highly specific service needs: “Emergency 24-hour AC repair that accepts credit cards.” Attribute-based searches: “Quiet coffee shops with reliable Wi-Fi and vegan options near downtown.” Optimizing the Core Pillars of Local AI Visibility Getting your clients into AI results requires a holistic approach that spans across owned media, earned media, and technical infrastructure. The following pillars form the foundation of an effective modern local SEO strategy. 1. Supercharging Your Google Business Profile (GBP) For Google Gemini and Google’s Search Generative Experience, the Google Business Profile remains the ultimate source of truth. However, simply filling out the basic info is no longer enough. To stand out to AI algorithms, you must leverage every feature available: Complete Every Single Attribute: From “wheelchair accessible restroom” to “identifies as veteran-owned,” select every relevant attribute. AI engines use these specific tags to filter results for highly specific user queries. Optimize the Business Description: Write a natural-sounding, descriptive business description that integrates your primary entities, services, and local landmarks without keyword stuffing. Regularly Update Google Updates (Posts): Keep your profile active with regular posts highlighting services, events, and offers. This signals to Google’s AI that the business is active and operational. Maintain an Accurate Product and Services Menu: Add detailed descriptions and pricing for your services and products directly within GBP. This structured data is easily parsed by AI models looking for specific offerings. 2. Implementing Advanced Schema Markup Schema markup (structured data) is the translator that helps AI search bots understand the exact meaning of your website’s content. Without proper schema, an AI might struggle to differentiate between a business’s phone number and a fax number, or its physical address and a mailing address. To optimize for AI visibility, go beyond basic LocalBusiness schema. Implement highly specific schemas such as: Dentist, Attorney, HVACBusiness, or Restaurant Schema: Use the most specific subtype available for your client’s industry. AreaServed Property: Clearly define the neighborhoods, zip codes, and cities the business serves to help AI engines understand geographic boundaries. KnowsAbout Property: Link your business or its founders to specific topics, certifications, or credentials to build topical authority. Product and Service Schema: Provide deep structured data on what the business sells, including pricing, availability, and customer reviews.

Google Search is undergoing its most significant architectural shift since the introduction of RankBrain. The transition toward an AI-first search engine, driven by the global deployment of “AI Mode”—including features like AI Overviews and conversational search—is accelerating at an unprecedented rate. Historically, expanding advanced search features to non-English languages and localized markets took years of engineering, manual translation, and region-by-region algorithmic tuning. Today, that entire paradigm has been disrupted. In a post-keynote interview with Indian broadcaster NDTV, Liz Reid, Google’s newly appointed Head of Search, revealed a crucial operational breakthrough: Google’s advanced multilingual AI models have dramatically simplified and accelerated the process of scaling AI features across different countries and languages. This shift marks a turning point not just for Google’s internal product roadmap, but also for international SEOs, global digital marketers, and content creators worldwide. Understanding Liz Reid’s NDTV Interview: A Paradigm Shift in Localization During the interview, Liz Reid highlighted how Google’s development of unified, multilingual large language models (LLMs) has transformed how the company approaches international rollouts. In the past, launching a major Google Search feature globally required localized product teams to build, train, and test custom models for each individual language. A system that worked perfectly in English might fail spectacularly when introduced to Hindi, Spanish, or Japanese due to differences in syntax, grammar, and cultural context. With Google’s new generation of multilingual models, the core AI architecture is inherently built to understand and process multiple languages simultaneously. According to Reid, this native multilingual capability means that when an AI feature is optimized and secured in one language, its underlying capabilities can be transferred to other languages and regions with far less manual friction. The technology is no longer constrained by a sequential, country-by-country rollout strategy; instead, it can scale almost globally in parallel. The choice of venue for this revelation is also telling. Speaking to NDTV, a major news outlet in India, highlights the strategic importance of multilingual markets. India is one of the most linguistically diverse countries in the world, with dozens of official languages and hundreds of dialects. For Google, proving that its AI Mode can accurately parse, summarize, and generate search results in this complex environment is the ultimate proof of concept for global scalability. The Science of Multilingual LLMs in Search To fully grasp why AI Mode can now scale so rapidly, it is essential to understand the underlying machine learning technology that powers Google’s current search stack. Traditional natural language processing (NLP) relied heavily on translation layers. When a user typed a query in a language like Vietnamese or Swahili, older systems would often translate the query into English, search the English index, retrieve the results, and translate those results back to the user’s native tongue. This process was slow, expensive, and highly prone to translation errors and loss of context. Modern Large Language Models, such as Google’s Gemini family, operate on a completely different principle. These models are trained on massive, multilingual datasets from day one. Instead of translating words, they map language to a shared, high-dimensional conceptual space (often referred to as vector embeddings). Cross-Lingual Transfer and Zero-Shot Learning One of the most powerful properties of these multilingual vector spaces is cross-lingual transfer. If an AI model learns a reasoning pattern or a factual relationship in English, that understanding naturally transfers to other languages it has been trained on, even if it has received very little direct training data in those specific languages. This is closely related to “zero-shot” or “few-shot” learning, where the AI can perform tasks in a new language with minimal to no language-specific training examples. For Google Search, this means that safety guardrails, summarization techniques, and factual verification algorithms developed for English-speaking markets can be rapidly deployed to dozens of other languages. The AI model does not need to relearn how to be safe, helpful, and accurate from scratch in every language; the core cognitive capabilities are already shared across its entire linguistic spectrum. Unified Semantic Understanding In practical terms, Google’s AI Mode does not see different languages as completely separate silos. Instead, it recognizes that a search query for “how to fix a leaky faucet” in English, “cómo arreglar un grifo que gotea” in Spanish, and “नल से पानी टपकना कैसे ठीक करें” in Hindi all share the exact same underlying user intent and conceptual meaning. By aligning these intents in a unified semantic space, Google can generate highly accurate, localized AI summaries drawing from a global pool of knowledge, while outputting the response in the user’s preferred language. Why “AI Mode” Scales Faster Than Traditional Search Features To appreciate the speed of the current AI rollout, we can compare it to the historical timelines of previous major Google Search feature launches: Google Lens: Launched initially in 2017, visual search took years to roll out globally, requiring extensive optimization for localized databases and regional device capabilities. Featured Snippets: First appearing around 2014, featured snippets required distinct programmatic algorithms for different language structures, leading to a staggered rollout that spanned several years. RankBrain and BERT: Google’s early deep learning integrations were launched first for English queries before being slowly customized and deployed to other languages over many months. In contrast, Google’s generative search features—such as AI Overviews (formerly known as the Search Generative Experience, or SGE)—have expanded to hundreds of countries and multiple languages in a fraction of that time. The transition from testing to global deployment has shrunk from years to months, and in some cases, weeks. Because the AI model handles the heavy lifting of linguistic adaptation natively, Google’s engineering teams can focus their efforts on localized compliance, product-market fit, and refining search quality, rather than rewriting the underlying search algorithms for every new country they enter. Implications for Global SEO and Content Creators The rapid, multilingual scaling of Google’s AI Mode has profound implications for digital marketing, search engine optimization, and global content strategies. Businesses can no longer treat international SEO as a secondary, delayed project. The AI-driven search experience

Brad Geddes on 20 Years of Paid Search Evolution The digital advertising landscape we navigate today is a highly sophisticated, multi-billion-dollar ecosystem dominated by artificial intelligence, automated bidding, and complex algorithms. However, this powerhouse industry did not appear overnight. It was forged through two decades of rapid trial, error, and paradigm shifts. Few people have witnessed and shaped this transformation as closely as Brad Geddes. An industry veteran, educator, and the creator of the Adalysis platform, Geddes began his journey in search engine optimization (SEO) back in 1996 and 1997, transitioning into paid search in 1998. After experiencing burnout in a completely different professional field, he taught himself website design and entered the nascent digital space as an at-home affiliate marketer for early internet pioneers like Amazon and eBay. Over the last two decades, Geddes has navigated every major shift in the search marketing landscape, establishing himself as one of the most authoritative voices in the pay-per-click (PPC) community. The True Inception of Pay-Per-Click: Goto.com While many modern marketers associate the birth of paid search with Google AdWords, the true pioneer of the pay-per-click model was Bill Gross, who launched Goto.com in 1998. This platform, which would later be rebranded as Overture and subsequently acquired to become Yahoo Search Marketing, introduced a revolutionary pricing model that changed the advertising world forever. Before Goto.com, digital advertising relied heavily on traditional media buying metrics, primarily CPM (cost per thousand impressions). Advertisers paid for eyes on a page, regardless of whether those users engaged with the content. Bill Gross turned this model on its head by placing a direct financial value on the click itself. For the first time, advertisers only paid when a user showed active intent by clicking on an ad. This auction-based system allowed businesses to bid openly for keyword rankings. If you wanted the top spot for a specific search query, you simply had to bid one cent more than your closest competitor. It was a transparent, simple, and highly effective model that laid the groundwork for the modern performance marketing industry. Google’s Rise to Dominance and the Changing Industry Culture It is easy to forget that Google was not always the undisputed king of search. In the early 2000s, Yahoo and Overture held massive market share, and advertisers were highly skeptical of Google’s entry into the space. In fact, Google did not firmly establish itself as the accepted industry leader until around 2006 or 2007. When Google first introduced its auction-based AdWords platform, advertisers initially disliked the system due to its complexity. Unlike Overture’s straightforward, high-bid-wins model, Google introduced Quality Score—a metric that combined bid price with click-through rate (CTR) and relevance. Furthermore, Google introduced the concept of “ad groups.” Instead of managing flat lists of keywords, marketers were forced to group related keywords and ads together. This structured approach required a significant shift in workflow, forcing marketers to transition from spending just a few hours a year on traditional advertising campaigns to managing digital campaigns on a weekly or even daily basis. Ultimately, advertisers accepted and adopted Google’s more complex platform for one simple reason: consumer behavior. Google’s superior, user-centric search engine attracted the vast majority of internet traffic. Marketers had to go where the users were. From Basement Operations to Corporate Giants Around the time Search Engine Land launched in 2006, the culture of the search industry underwent a massive evolution. In the early days, the PPC and SEO communities were tight-knit and highly collaborative. Digital marketing was run largely by hobbyists, affiliate marketers, and small agencies operating out of spare bedrooms and basements. As search engines began to prove their immense profitability, the industry rapidly shifted into a mainstream corporate environment. This transition was fueled by massive infusions of venture capital money, skyrocketing corporate salaries, and lavish industry parties, including famous, over-the-top private yacht parties. However, this corporate maturity came with a trade-off. In the early years, search professionals openly shared their tactics, tests, and data with one another. As corporate legal departments took over and non-disclosure agreements (NDAs) became the industry standard, this open-source culture of information sharing largely faded, replaced by highly guarded proprietary strategies. Major Milestones That Changed PPC Forever Reflecting on the timeline of search marketing, Geddes points to several critical turning points that permanently altered the trajectory of the industry. The Separation of SEO and Paid Search In the early days of search, digital marketers were generalists who managed both organic search and paid campaigns. That all changed when Google rolled out its major organic algorithm updates: Panda, Penguin, and Pigeon. Panda: Targeted low-quality content and thin affiliate sites. Penguin: Penalized manipulative link-building schemes. Pigeon: Completely restructured localized search results. These updates made organic SEO incredibly complex and technical. Marketers realized they could no longer divide their attention between the two channels and maintain high performance. The industry fractured, forcing professionals to specialize as either SEO experts or dedicated paid search practitioners. The Dawn of Automated Bidding The second major milestone was the development and successful implementation of automated bidding. Before automation, bid management was a tedious, highly manual process. Marketers spent hours export-importing data, running complex Excel formulas, and manually adjusting bids for hundreds of thousands of keywords. When search engines introduced reliable machine learning algorithms capable of predicting conversion probability in real-time, it freed up massive amounts of time for advertisers. Instead of performing administrative data entry, PPC managers could finally focus on high-level strategy, creative copywriting, and deep conversion rate optimization (CRO). The 2005 Domain Policy Shift In 2005, Google made a structural decision that fundamentally disrupted the affiliate marketing industry: they instituted a policy allowing only one ad per domain to appear on a search engine results page (SERP). Prior to this change, multiple affiliate marketers could bid on the same keyword and direct traffic straight to the merchant’s URL using their affiliate tracking links. Google’s search results were often cluttered with identical destinations. The new policy forced affiliate marketers to build their own dedicated

Google adds AI shopping visibility insights to Merchant Center The landscape of e-commerce search is undergoing its most significant transformation in a generation. Consumers are moving away from rigid, keyword-based search queries and transitioning toward conversational, intent-driven interactions with artificial intelligence. To help retailers navigate this shift, Google is rolling out new AI performance insights inside Google Merchant Center. These analytical tools are specifically designed to help brands track, measure, and optimize how their products appear across Google’s expanding array of AI-powered shopping experiences. As platforms like Google Gemini and AI Overviews increasingly dictate consumer discovery, understanding product visibility in these environments has become a critical priority for digital marketers. The new reporting tools within Merchant Center aim to demystify how Google’s algorithms index, rank, and present products during conversational shopping journeys. The Shift to Conversational Commerce and Generative AI For years, e-commerce search followed a predictable pattern. A user typed a query like “men’s leather running shoes size 10,” and the search engine returned a list of products matching those exact keywords. Today, search is becoming highly contextual, iterative, and conversational. A shopper might now ask Google Gemini: “I’m training for a marathon, have slightly flat feet, and prefer sustainable materials. What are some highly-rated running shoes under $150 that fit this description?” To answer such highly specific queries, Google’s AI must synthesize a massive amount of structured and unstructured data. It pulls information from merchant product feeds, user reviews, editorial guides, and manufacturer specifications. If a retailer’s product feed lacks the granular detail needed to satisfy these specific parameters, that product simply will not appear in the AI’s recommendations. Google’s introduction of AI shopping visibility insights addresses this exact challenge. By providing direct feedback on how products are performing within generative AI surfaces, Google is giving merchants a diagnostic roadmap to improve their discoverability in a conversational search ecosystem. Key Features of the New AI Performance Insights The update to Google Merchant Center introduces four primary analytical reporting tools. Each focuses on a different aspect of the AI-driven customer journey, offering a combination of competitive benchmarking and diagnostic feedback. 1. Share of Voice Insights In traditional Search Engine Optimization (SEO) and Pay-Per-Click (PPC) advertising, Share of Voice (SOV) measures your brand’s exposure compared to the total addressable market. In generative AI search, however, tracking SOV is much more complex. AI Overviews and conversational interfaces typically recommend a highly curated selection of products—often just three or four top options—rather than pages of search listings. The new Share of Voice insights benchmark your brand’s visibility directly against similar retailers within these AI-curated carousels and summaries. This allows merchants to see if they are winning the digital shelf in generative search results or if competitors are capturing the majority of AI-driven recommendations for key product categories. 2. Shopping Funnel Performance Reports Consumer journeys within AI shopping environments do not always follow a linear path. Users often move back and forth between exploring options and narrowing down choices. To help merchants understand this behavior, the new reporting suite breaks down funnel performance into three distinct stages: Discovery: How often your products appear when users are starting their search or asking broad, category-level questions. Evaluation: How your products perform when users are actively comparing different brands, reading synthesized reviews, or asking the AI to weigh pros and cons. Purchase: The frequency with which your products are featured as the final recommended option when the user is ready to make a transaction. By analyzing these stages, retailers can pinpoint exactly where they are losing potential customers. For example, if a brand has high visibility during discovery but drops off during evaluation, it may indicate a need to improve product reviews or address negative sentiment that the AI is detecting across the web. 3. Product Term Insights Understanding how people talk to AI is crucial for modern product feed optimization. Product term insights show the actual conversational search queries that consumers are using when discovering a merchant’s products. These terms differ significantly from traditional short-tail keywords. They often include long-tail phrases, natural language questions, and highly specific modifiers regarding use cases, aesthetics, or values (e.g., “cruelty-free waterproof mascara for sensitive eyes”). Having access to this query data allows marketers to adjust their product titles, descriptions, and landing page content to align more closely with real-world conversational search behavior. 4. Product Attribute Insights Perhaps the most actionable part of the update is the product attribute insights report. AI models rely heavily on structured attributes—such as color, material, style, sizing standards, and age group—to filter and match products to user requests. If these attributes are missing or incomplete in your Google Merchant Center feed, your products may be excluded from relevant conversational results. The product attribute insights tool automatically scans a retailer’s product feed to identify missing, incomplete, or poorly formatted specifications. It then highlights which attributes should be added or optimized to increase the likelihood of the product being recommended by Google’s AI. Why AI Visibility Matters for Retailers and Brands For years, Google Merchant Center served primarily as a backend repository—a tool to upload product catalogs, manage pricing, and feed data into Google Shopping Ads. However, the platform is steadily transforming into an active AI commerce optimization platform. This change is driven by the reality that search visibility is no longer just about bidding strategies; it is about data completeness and contextual relevance. As Gemini and AI Overviews become the default entry points for many online shoppers, organic and paid visibility are merging in unique ways. In an AI-generated product comparison, Google does not merely present an ad; it explains *why* a product is a good fit for the user’s specific request. If your product feed lacks the structured data to support those explanations, your brand remains invisible. By offering early access to these performance metrics, Google is giving proactive brands a significant first-mover advantage. Retailers who utilize these insights to clean up their feeds and align their content with conversational trends will be

Vanessa Fox on the birth of Google Search Console For modern search engine optimization (SEO) professionals, Google Search Console (GSC) is an indispensable daily tool. It is the primary channel through which Google communicates directly with site owners, offering critical insights into indexing status, search traffic, manual actions, and technical errors. Yet, there was a time when this bridge between Google and the webmaster community did not exist. Google was a black box, and SEO was largely a game of trial, error, and speculation. That dynamic changed forever thanks to Vanessa Fox. As the creator and driving force behind what was originally known as Google Webmaster Tools, Vanessa transformed how the search engine giant interacts with the web ecosystem. In an in-depth retrospective interview, Vanessa shared insights into her journey at Google, her work with Matt Cutts, her transition to Search Engine Land, and her perspective on the modern, AI-driven state of search engine optimization. The Genesis of Google Search Console: From XML Sitemaps to Webmaster Tools To understand the creation of Google Search Console, one must look back to the early 2000s. At the time, search engines crawled the web using basic link-following algorithms. If a page was not linked to by another indexed page, it remained invisible to search engines. For new websites or those with complex database-driven architectures, getting indexed was a major hurdle. Vanessa Fox joined Google with a professional background in user experience (UX) and technical writing. This unique combination of skills allowed her to view search engine indexing not just as a database engineering challenge, but as a communication and accessibility issue for site owners. The solution began with the introduction of XML Sitemaps. Initially, Google launched the Sitemaps protocol as a way for webmasters to provide a direct list of URLs they wanted crawled. However, the feedback loop was entirely one-sided; webmasters submitted their files but received no confirmation of whether the URLs were successfully processed, ignored, or blocked by technical errors. Vanessa recognized that submission was only half the battle. Webmasters needed feedback. This realization led to the birth of Google Webmaster Tools. Under Vanessa’s guidance, the platform was developed to show crawl errors, index status, and query data. What started as a simple dashboard to support XML Sitemaps eventually evolved into the robust, multi-featured Google Search Console we rely on today. Inside the Early Days of Google: The Kirkland Office and 200 Employees When Vanessa joined Google, the company was a fraction of the size it is today. She worked out of the Kirkland, Washington office, a regional hub that felt distinct from the massive Mountain View headquarters. At that time, Google employed approximately 200 people worldwide. This small-scale environment allowed for cross-departmental collaboration that would be nearly impossible in today’s corporate landscape. Vanessa worked closely with engineering teams and search quality representatives, most notably Matt Cutts, who was then the public face of Google’s search spam team. Vanessa and Matt collaborated to bridge the gap between internal search engineering and external webmaster frustration. They turned to Google’s internal help center data to analyze where site owners were struggling. If thousands of users were submitting help requests about a specific crawling error, Vanessa’s team worked to build that diagnostic data directly into Webmaster Tools, turning reactive support into proactive self-service diagnostics. A Regrettable Financial Decision: Selling Google Stock Options Too Soon Working at a hyper-growth startup like Google in the mid-2000s came with substantial financial upside, particularly in the form of stock options. However, navigating those options was risky business for employees who had lived through the volatility of the dot-com bust. During her interview, Vanessa shared what she describes as a “sad story” regarding her Google stock options. Prior to her tenure at Google, she had worked at AOL, where she witnessed firsthand how quickly a tech giant’s stock could plummet. Haunted by that experience and wishing to avoid a similar financial setback, Vanessa decided to sell her Google stock options shortly after they vested. While the decision made practical sense based on her past experiences in the tech industry, she admits to selling far too early. Had she held onto those options, their value would have increased exponentially alongside Google’s rise to a multi-trillion-dollar market cap. It is a relatable cautionary tale of the unpredictability of the early tech sector. Leaving Google and Joining Search Engine Land In 2007, Vanessa made the difficult decision to leave Google. Having successfully established Webmaster Tools as an essential piece of search infrastructure, she was ready for new professional challenges. Shortly after her departure, she joined the editorial team at Search Engine Land. The publication, co-founded by Danny Sullivan, was fast becoming the leading source of news and analysis for the search marketing industry. Vanessa brought a highly technical, internal-facing perspective to the site, translating complex algorithmic concepts into actionable advice for search marketers. Her columns demystified how Google crawled, indexed, and processed information, helping to professionalize the SEO industry during its formative years. Debunking SEO Misconceptions and Managing the Panda Era Throughout her career, Vanessa has been a vocal opponent of manipulative “black hat” SEO techniques, advocating instead for technical health and user-centric design. In the early days of search, many marketers viewed Google as an adversary to be tricked. There was a widespread misconception that the Google spam team spent their days manually penalizing individual websites out of spite or bias. Vanessa helped debunk these myths by explaining how Google’s engineering team actually functioned. The spam team’s primary goal was to write scalable algorithmic rules to filter out low-quality content, not to play a game of whack-a-mole with individual site owners. This algorithmic approach to search quality culminated in major core updates, most notably the Google Panda update in 2011. Vanessa spent years conducting Panda SEO audits to help affected businesses recover. She notes that Panda shifted the paradigm of SEO because it analyzed site-wide quality rather than page-level metrics. If a website hosted a massive volume of thin, duplicate,

The Evolution of Customer Acquisition in Google Ads In digital marketing, finding new customers is the lifeblood of sustainable growth. However, many advertisers face a common and frustrating challenge: ad spend that is meant to drive expansion often gets funneled into reaching users who are already familiar with the brand. Retargeting loops and brand-biased algorithm optimizations can inflate performance metrics on paper while delivering very little incremental value. To address this challenge, Google is expanding its suite of New Customer Acquisition tools. The platform is introducing a dedicated “new prospects” targeting mode designed specifically to help advertisers find and convert consumers who have had absolutely no prior contact with their business. This new targeting capability marks a significant shift in how automated Google Ads campaigns balance mid-funnel conversion with upper-funnel discovery. What Is the “New Prospects” Mode? The “new prospects” mode is an advanced layer of Google’s New Customer Acquisition framework. Historically, digital advertising platforms defined “new customers” based primarily on transaction history. If a user had not purchased from a brand within a specific window, they were classified as a new customer, regardless of how many times they had visited the website, read the blog, or watched the brand’s videos on YouTube. The “new prospects” mode changes this paradigm. Instead of focusing solely on the transaction boundary, this new setting isolates “cold” audiences who are entirely unaware of the brand. By programmatically filtering out warm leads, Google allows advertisers to target their budgets exclusively toward genuine brand discovery. How “New Prospects” Differs from Standard Customer Acquisition To understand the utility of this new mode, it helps to contrast it with existing options within Google Ads: Standard Campaigns: Target any user predicted to convert, often prioritizing past purchasers and highly warm leads because they offer the path of least resistance to a conversion. New Customer Acquisition (Value Mode): Bids more aggressively for new customers by adding an extra valuation layer to non-purchasers, but still allows ads to serve to existing customers if they are highly likely to buy. New Customer Acquisition (New Customer Only Mode): Limits bids strictly to users who have not made a purchase recently, though these users may still be highly familiar with the brand. New Prospects Mode: Excludes not only past buyers but also any user who has shown brand awareness through search behavior, website visits, app engagement, or social interaction. The Core Mechanics: How Google Excludes Warm Audiences The effectiveness of cold-audience targeting relies heavily on the accuracy of audience exclusions. To ensure ads only reach completely unaware prospects, Google’s system automatically identifies and filters out users who have taken any of the following actions: 1. Purchased Previously Google cross-references first-party data, Customer Match lists, and conversion tracking pixels to identify existing buyers. Anyone with a record of purchase activity is systematically excluded from the targeting pool. 2. Searched for Brand Terms If a user has recently searched for the brand’s name, product-specific names, or associated trademark terms, they are flagged as brand-aware. This prevents the “new prospects” mode from bidding on users who are already actively navigating toward the brand via search. 3. Visited a Website or App Using data from Google Analytics 4 (GA4), global site tags, and SDK integrations, the system identifies and excludes historical visitors to the advertiser’s digital properties. Whether a user read a blog post six months ago or abandoned a cart last week, they are excluded from the prospecting pool. 4. Engaged with Brand Content Across Google and YouTube Because Google’s ecosystem spans across Search, Maps, Google Play, and YouTube, the platform can track soft engagement indicators. Users who have watched a brand video on YouTube, subscribed to a channel, or interacted with other owned media assets are excluded to maintain the integrity of the cold audience. Why Cold Audience Targeting Matters for Modern Brands For years, digital marketers have relied heavily on performance marketing tactics that harvest existing demand. While this approach yields high Return on Ad Spend (ROAS) in the short term, it eventually leads to a performance plateau. Once a brand exhausts its high-intent warm audience, the cost per acquisition (CPA) rises, and growth stalls. By offering a dedicated mechanism to reach cold audiences, Google is helping advertisers automate the top-of-funnel discovery process. This shift provides several strategic benefits: Incremental Growth Over Audience Cannibalization Many PPC campaigns suffer from audience cannibalization, where paid search ads capture conversions that would have occurred organically. By excluding users who search for brand terms or have visited the site, the “new prospects” mode ensures that every dollar spent is buying truly incremental reach rather than subsidizing organic traffic. More Efficient Budget Allocation When launching broad targeting or brand-building campaigns, budget is often wasted on users who are already deeply loyal to the brand. Isolating cold audiences ensures that top-of-funnel creative assets are displayed only to people who actually need to see them to learn about the brand. Nurturing the Modern Customer Journey The path to purchase is rarely linear. Consumers frequently discover a brand on YouTube, research it on Search, and buy weeks later. Introducing a brand to a consumer early in this journey allows advertisers to build trust before the consumer reaches the high-competition, high-cost comparison phase of their search. Analyzing the Financial Impact: Value-Based Bidding and ROAS The business case for cold-audience prospecting becomes clearer when analyzed alongside Google’s value-based bidding (VBB) strategies. Scaling cold traffic can sometimes lead to a temporary drop in nominal ROAS because cold prospects convert at a lower rate than warm leads. However, Google’s data shows that when configured properly, customer acquisition modes can yield substantial efficiency gains. According to Google, advertisers who leverage the New Customer Acquisition Value Mode see an average 9% improvement in ROAS. This lift is achieved by assigning a higher value to new buyers within smart bidding algorithms—specifically by valuing new customers at twice the average order value (AOV). When bidding engines are told that a new customer is worth twice as much as a repeat

For years, search engine optimization has been governed by checklists. Marketers spend hours optimizing title tags, tweaking meta descriptions, structuring internal links, and ensuring that keyword density hits the sweet spot. While this technical discipline is essential for earning visibility, it often leaves a critical gap: what happens after a user actually clicks through to your site? With the rise of search engine updates, AI Overviews, and zero-click searches, organic visibility is increasingly hard to secure. Ranking on the first page of Google is no longer the final victory. If your content fails to inspire action once a user arrives, your search traffic is merely a vanity metric. To turn passive readers into active customers, SEO content must transition from merely informative to deeply persuasive. This challenge is not unique to written media. Modern social commerce, particularly among top creators on platforms like TikTok Shop, has mastered this transition. These creators do not rely on massive, pre-existing follower bases to drive millions of dollars in sales. Instead, they leverage fundamental principles of consumer psychology to capture attention and inspire immediate action. By applying these exact psychological frameworks to written SEO content, digital marketers can dramatically improve conversion rates and maximize the value of every organic visit. The TikTok Shop Conversion Formula To understand how persuasion works in the modern digital landscape, it is helpful to look at platforms where transaction speeds are fastest. On platforms like TikTok Shop, affiliate creators are generating extraordinary sales volumes from audiences who have never heard of them before. In these environments, success is not a function of celebrity status; in fact, the vast majority of views on high-converting product videos come from discovery algorithms rather than direct followers. The success of these creators is driven by a repeatable, psychology-based formula. When analyzed closely, this formula consists of four core components: Visual Hooks: Immediately interrupting the user’s scroll to secure the first few seconds of attention. Psychological Levers: Activating subconscious human desires and pain points to establish relevance. Authentic Storytelling: Framing the product or service within a relatable human narrative rather than a dry list of specifications. Relentless Testing: Iterating on hooks, angles, and calls to action to find the precise combination that converts. This exact structure can be translated directly into long-form written content. Instead of a visual hook, a blog post uses a compelling hook in the introduction. Instead of a video narrative, it uses structured, empathetic copy that addresses the reader’s immediate reality. The underlying engine remains identical: a deep understanding of human decision-making. The Core Principle: Emotional Decisions, Rational Justification A common mistake in content marketing is assuming that consumers make logical, analytical decisions. Traditional product copy often focuses heavily on features, integrations, technical specifications, and pricing tiers. While these details are necessary, they rarely inspire the initial decision to buy. Neuroscience and behavioral economics demonstrate that humans make decisions emotionally and then look for logical arguments to justify those decisions after the fact. If your content only speaks to the analytical mind, it misses the subconscious triggers that drive action. Persuasive content must speak directly to the emotional and biological motivations of the reader, providing the technical details as supporting evidence to validate their emotional choice. The Eight Primary Desires in Sales Psychology Human behavior is guided by a set of foundational biological and social desires. Often referred to in advertising psychology as the core drivers of human motivation, these eight desires are hardwired into our biology. They cross cultural, geographic, and generational lines. By aligning your SEO content with one or more of these core desires, you transform your copy from a passive explanation into an active persuasive force. 1. Care and Protection of Loved Ones The instinct to protect, nurture, and secure the well-being of family and loved ones is one of the strongest emotional triggers in existence. When a purchase decision is framed around the safety and future of those who depend on us, the motivation to act increases exponentially. To apply this to your written content, move beyond listing what your product does and focus on the security it provides to the user’s circle of care. For example: In Insurance Copy: Instead of focusing solely on policy limits and premiums, emphasize peace of mind: “When the unexpected happens, having the right coverage ensures your family can maintain their home and lifestyle without financial strain during an already difficult time.” In Home Security Systems: Shift the focus from sensor technicalities to emotional safety: “A security event isn’t just about lost property; it’s about a compromised sense of safety. The right system preserves your family’s peace of mind.” In Residential Services: Frame maintenance as preservation: “Addressing a minor roof leak today means protecting the structural integrity of the home your family relies on every single day.” 2. Survival, Enjoyment of Life, and Life Extension Humans possess an innate drive to live longer, healthier, and more vibrant lives. This lever goes beyond basic medical survival; it encompasses physical vitality, energy, and the ability to enjoy life’s experiences without physical limitations. When writing about health, wellness, fitness, or lifestyle products, connect your offerings directly to active lifestyle preservation: For Nutritional Supplements: “By supporting your cellular health, you are preserving the energy needed to hike your favorite trails, play with your children, and feel fully present in your daily life.” For Outdoor and Recreational Gear: “Engineered to withstand the elements, this gear ensures you can focus entirely on the horizon ahead, not on whether your equipment will hold up.” For Travel and Leisure: “Taking time to disconnect isn’t just a luxury; it is a vital reset that supports mental clarity and long-term well-being.” 3. Enjoyment of Food and Beverage Food and drink are more than basic survival requirements; they represent pleasure, sensory indulgence, comfort, culture, and social connection. Content that appeals to these sensory experiences can evoke physical desire and nostalgia. Use evocative language that helps the reader taste, feel, and experience the offering through the screen: For Meal Kit

The rise of generative artificial intelligence has triggered an unprecedented land grab for high-quality digital data. As tech giants and AI startups race to build increasingly sophisticated large language models (LLMs), they require massive volumes of human-generated text to train their algorithms. While book archives, scientific papers, and news articles have all played their part, one platform has emerged as an absolute cornerstone of the AI revolution: Reddit. Reddit CEO Steve Huffman recently made headlines by asserting that modern LLMs “would not exist” without the platform’s vast repository of human conversation. Speaking on the critical role that user-generated content plays in machine learning, Huffman described Reddit’s data as “modern oil” for the AI era. His comments highlight a dramatic shift in how the tech industry values digital conversations, moving away from an open-web philosophy toward a highly monetized, heavily guarded data marketplace. As Reddit secures multi-million dollar partnerships with industry leaders like Google and OpenAI while simultaneously threatening legal action against unauthorized data scrapers, the rules of the internet are being rewritten. Here is a deep dive into why Reddit’s data is so vital to AI, how the platform is capitalizing on its digital goldmine, and what this means for the future of the internet. Why Reddit Data is the “Modern Oil” of AI Training To understand why Steve Huffman claims LLMs owe their existence to Reddit, one must understand how machine learning models learn to speak like humans. AI models do not understand language in the way humans do; instead, they analyze patterns, probabilities, and context across trillions of words. The quality of the output is directly dependent on the quality and diversity of the training input. For years, AI developers relied on web scraping to gather training data. However, much of the internet consists of sterile product descriptions, repetitive SEO blogs, or highly structured academic texts. These sources do not reflect how humans actually talk to one another in everyday life. Reddit offers something entirely different. It is a living, breathing archive of human interaction. With over 100,000 active communities (subreddits) covering everything from niche technical troubleshooting to emotional support, creative writing, and political debate, Reddit provides an unparalleled look into authentic human communication. Here is why Reddit data is uniquely valuable to AI development: Conversational Nuance: Unlike static articles, Reddit threads show how conversations flow. AI models learn slang, sarcasm, humor, disagreement, and empathy by analyzing how users respond to one another. The Power of Upvotes and Downvotes: Reddit’s built-in moderation system acts as a natural quality filter. When users upvote helpful or entertaining comments and downvote spam or misinformation, they are effectively labeling the data for machine learning algorithms. AI developers can use these signals to train models on what constitutes a “good” or “bad” response. Real-Time Information: Reddit is often the first place news breaks, trends start, and software bugs are solved. It serves as a real-time pulse of human activity, making it invaluable for keeping AI models current. Niche Expertise: From coding advice on r/programming to financial discussions on r/wallstreetbets, Reddit hosts specialized knowledge that is difficult to find consolidated anywhere else on the web. Without this massive, diverse, and naturally moderated dataset, the conversational fluidity of modern chatbots like ChatGPT or Claude would likely be far more robotic and far less capable of understanding complex human queries. The Lucrative Partnerships: Google and OpenAI Recognizing the immense value of its data, Reddit has transitioned from a platform that allowed free, unchecked access to its API to one that demands premium compensation. This shift has resulted in massive licensing agreements with the biggest players in the AI space. The Google Partnership In early 2024, Reddit signed a landmark data-sharing deal with Google, valued at approximately $60 million annually. Under this agreement, Google gained real-time access to Reddit’s data API, allowing the search giant to train its Gemini models on up-to-the-minute discussions. Additionally, this deal paved the way for Reddit threads to be featured more prominently in Google search results, transforming how users discover forums online. The OpenAI Partnership Shortly after the Google deal, Reddit announced a major partnership with OpenAI. This collaboration allows OpenAI to integrate Reddit content directly into ChatGPT and other upcoming products. It also enables OpenAI to utilize Reddit’s data APIs to continuously train and refine its LLMs. In return, Reddit is incorporating OpenAI’s advanced AI features into its own platform for both users and moderators. These partnerships have fundamentally validated Reddit’s business model following its initial public offering (IPO) in early 2024. By turning its archive of human conversation into a recurring revenue stream, Reddit has proven that user engagement can be monetized far beyond traditional display advertising. The War on Scraping: Why Some AI Firms Face Lawsuits While Google and OpenAI have agreed to pay for Reddit’s data, not everyone in the AI sector has been willing to play by the rules. For years, AI research labs and tech startups scraped the web indiscriminately, operating under the assumption that public data was free for the taking. This practice is known as “web scraping” or “web crawling.” Steve Huffman has made it clear that the era of free, unauthorized data harvesting is over. Reddit has updated its robots.txt file—the standard web protocol that tells automated bots which parts of a site they are allowed to visit—to block unauthorized AI crawlers. The platform has also implemented strict rate limits and paywalls on its API. Huffman has defended this aggressive stance, explaining that companies scraping Reddit without permission are effectively stealing intellectual property and undermining the platform’s value. He noted that Reddit is actively tracking unauthorized scrapers and is prepared to use legal means to protect its assets. Some AI companies, particularly those that refuse to negotiate licensing agreements but continue to bypass technical blocks, now face the very real threat of costly intellectual property lawsuits. The message from Reddit is clear: if you want to build commercial AI products using the collective knowledge of Reddit’s users, you must pay for the