Examining New Research, “AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence,” for SEO/GEO Insights (Hamsterdam Research)

By Ethan Lazuk

Welcome to another edition of Hamsterdam Research! 🐹

This is where we look at recent AI research papers to learn what they’re talking about and explore their hypothetical implications for the future of search and SEO/GEO strategies.

This time we’ve got a paper of pretty direct relevance for our field. We’ll look at research called, “AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence.”

It was published on August 18, 2026, and its authors are Stephanie T. Wang, Jeffrey Gleason, Yakov Bart, Christo Wilson, and Danaé Metaxa.

Why should SEO/GEO professionals care about this research paper, beyond the obvious title?

What makes this paper valuable for SEO/GEO is that it gives us a rare causal look at how the search interface changes search behavior itself, not merely whether AI Overviews or AI Mode correlate with lower CTR.

Here are some main takeaways:

• AI Mode reduced external click-through by 18.8 percentage points. Removing AI features increased click-through by 8.8 points.
• AI Mode reduced clicks to news sites, Reddit, and Wikipedia.
• Users spent more time per search session in AI Mode, but had fewer search sessions overall.
• AI Mode did not increase question-style queries or searches per session.
• AI Mode reduced trust, satisfaction, usefulness, agency, and perceived relevance/personalization.
• Users complained about fewer/diverse sources and difficulty reaching specific websites.
• AI Mode increased use of competing search engines by 11.2 percentage points.

Perhaps the biggest SEO/GEO implication: AI search keeps more attention inside the search interface, so visibility and traffic are becoming less tightly connected.

Let’s start by reviewing the paper’s abstract.

Here’s the abstract (with my highlights):

“The integration of generative AI into web search delivers synthesized answers to user queries, changing how people navigate and assess information, while raising concerns about the downstream impacts on publishers who supply the underlying content. We conduct a preregistered field experiment (N=1,100) on Google Search, the dominant online search platform, to estimate the causal effects of AI Overviews and AI Mode on user behavior, perceptions, and publisher traffic. We show that removing AI Overviews and AI Mode increases click-through rates to publishers, while an AI Mode-only experience reduces click-through rates and erodes user experience and trust in information found on Google. These findings show that integrating generative AI into web search reshapes online attention, with economic consequences for the online publishers that sustain both search platforms and the overall information ecosystem.”

Next, let’s break down the paper’s key vocabulary terms:

• Good abandonment: When a user gets what they need directly from the search results and doesn’t click a website. (Remember, a “zero-click” search isn’t automatically a failed search.)
• Substitution: When AI search replaces an activity that would otherwise have happened, such as visiting a publisher or using traditional search. (The paper contrasts this with “complementarity.”)
• Complementarity: When AI search supplements traditional search rather than replacing it, potentially causing users to search more or visit additional websites.
• Click-through rate (CTR): The proportion of searches that result in a click to an external website.
• Publisher referral traffic: Visits that publishers receive when a search engine sends users to their websites.
• Search engine substitution: Users moving from one search engine to another because of changes to the search experience. The study measured use of Bing, DuckDuckGo, and Yahoo after exposure to AI Mode.
• User agency: The user’s perceived ability to control how they search, evaluate sources, and reach the information they want. This is relevant because AI Mode reduced perceived agency.

Based on the paper, I’d say the most useful terms are “good abandonment,” “substitution,” “complementarity,” “publisher referral traffic,” and “user agency.”

Bueno. Let’s take a deeper look at the research paper’s contents now.

If you want to follow along, you can grab a PDF or the HTML version on arXiv.

The full paper has 10 sections.

We’ll summarize the main ones below.

1. Introduction

Before the introduction, the researchers have a “Significance statement,” in which they say:

“In a preregistered randomized controlled trial with participants using Google Search in their everyday browsing, we find that removing AI features in Google Search increases clicks to third-party publishers, while Google’s conversational AI search reduces clicks and worsens user experience. Our findings show how integrating generative AI into search can reshape how users engage with the broader web, providing more information directly in search while reducing engagement with external sources, with implications for the economic sustainability of online publishers.”

Within the introduction itself, the researchers provide more context for their study, speaking about the introduction of AI Overviews in 2024 and AI Mode in 2025.

Then they get into interesting topics around SERPs: “The design of the search engine results page (SERP) matters because it shapes how users assess the credibility of information sources [29] and distributes clicks and attention between third-party publishers and Google’s own properties”

They continue, discussing the dynamics of the user-Google-publisher relationship:

“The introduction of AI into search has raised these stakes and increasingly strained the relationship between Google and third-party publishers: publishers view AIO and AI Mode as substitutes for their content and report falling referral traffic from Google Search [41, 3, 5, 7]. Google disputes these claims, arguing that AI features are complements that have kept overall referral traffic steady, improved click quality, and expanded opportunities for websites to be surfaced through longer, more complex queries [38]. On the other hand, users may benefit from the convenience of having their questions answered directly on the search results page in what is known as “good abandonment” [9, 26], creating a three-way tension in which user convenience, publisher sustainability, and Google’s consolidation of informational authority do not easily align.”

They further explain how “This dynamic of AI substituting for third-party content is not unique to search” but also pertains to AI assistants like ChatGPT, citing losses in views to Stack Overflow and Wikipedia after its launch.

In terms of existing research, they note how “AI Mode, Google’s fully conversational search experience, remains understudied.”

Hence, their study:

“Existing work documents meaningful consequences of generative AI in search for users, publishers, and platforms, but relies primarily on observational data and controlled laboratory experiments. How, and to what extent, generative AI in search shapes user behavior, perceptions, and publisher traffic in naturalistic, real-world settings remains an open question. In particular, it is unclear whether AI in search primarily facilitates users’ discovery of third-party publishers or increasingly substitutes for it. To address this gap, we conduct a preregistered field experiment [2] in which participants are randomly assigned to one of three conditions via a browser extension [46, 24, 36] that manipulates the availability of AI in search features [17] and captures behavioral and attitudinal outcomes: 1) No AI Search, which hides AI Overviews and AI Mode; 2) Current Search, which makes no changes; and 3) AI Mode Search, which redirects all searches to AI Mode.”

Thus, the significance of this research is that it isn’t merely “observational data” or a controlled lab experiment, but takes place in a real-world setting where users either get no AI search, a normal experience, or redirected to AI Mode.

2. Experimental Methods and Data

The researchers discuss the make-up of their study:

“We limited recruitment to US-based participants, and screened for individuals who (1) were 18 years or older, (2) used Google Chrome as their primary browser, and (3) used Google Search as their primary search engine. After giving informed consent, participants completed a pre-survey (Section S7.2) that elicited their familiarity with and sentiment towards LLM applications, as well as perceived trust, usefulness, satisfaction, agency, and personalization/relevance of information-seeking on Google. Participants then installed our browser extension and experienced three days of the baseline, Current Search condition. This period allowed us to measure pre-treatment versions of behavioral outcomes like search sessions per day and click-through rate.“

That sets the stage. Now the researchers conducted their study:

“After three days, participants were randomly assigned to one of our three search conditions: (1) No AI Search, (2) Current Search, or (3) AI Mode Search. The No AI condition hid Google’s AI Overviews, including those at the top of the results page, in the middle of the results page, and nested within People Also Ask components. AI Mode searches were also redirected to general search. The Current Search condition made no modifications to Google Search: AI Overviews may be present and AI Mode is accessible. The AI Mode condition redirected all searches to AI Mode. Participants experienced their assigned conditions for seven days. At the end of the seven-day period, we sent participants a post-survey that asked about perceptions of information-seeking on Google (Section S7.3) and presented head-to-head comparisons of AI Mode and Current Search responses in the context of news queries (Figure S1).”

A “total of 1,444 participants” were enrolled. “Of these, $N=\mathrm{1,100}$ made at least one search during the 7-day experiment period and were invited to take the post-experiment survey,” they write.

3. Results

This section is fairly detailed, so in lieu of quoting from the researchers, I’ll break down their findings into snackable sections:

1. AI Search Reduced Clicks to the Open Web

When participants were forced into AI Mode, external click-through rate fell by 18.8 percentage points. When AI features were removed, click-through increased by 8.8 points.

2. The Traffic Loss Affected Major Types of Publishers

AI Mode reduced the share of users clicking:

• News sites by 12.5 percentage points
• Reddit by 21.2 points
• Wikipedia by 9.9 points

3. AI Mode Changed How People Searched

Users had 0.92 fewer Google search sessions per day in AI Mode, yet the sessions they did have lasted about 0.43 minutes longer.

That said, AI Mode did not significantly increase searches per session or question-form queries, such as “how,” “why,” or “what.”

4. AI Mode Made Some Users Look Elsewhere

AI Mode increased the share of users searching on Bing, DuckDuckGo, or Yahoo by 11.2 percentage points.

It also increased stated intention to switch to Bing by 1.25 points on a seven-point scale.

5. Users Generally Liked the AI Mode Experience Less

AI Mode significantly reduced:

• trust
• usefulness
• satisfaction
• user agency
• perceived personalization/relevance

The personalization/relevance result is especially interesting because the researchers expected AI Mode to improve it, but the opposite happened.

6. Users Explained What They Disliked

Among AI Mode participants:

• 17.6% mentioned reduced control or agency
• 15.3% had difficulty reaching specific websites
• 13.4% complained about limited links or source diversity
• 6.2% thought responses were too verbose
• 4.6% raised accuracy or hallucination concerns

There were positives too: 14% praised efficiency/time savings and 6.2% mentioned good summarization.

Caveat: The No AI Search results should be interpreted more cautiously than the AI Mode results. Google’s HTML changed during the experiment and broke the researchers’ extension, so only about 51% of AI Overviews were successfully hidden overall.

4. Discussion

This study “provides causal evidence about the effects of integrating generative AI into web search.”

The main finding is somewhat jarring:

Leveraging an in-situ browser experiment that manipulated users’ access to AI Overviews and AI Mode in Google Search over a 10-day study period, we show that the current trajectory of AI-mediated search presents a clear tradeoff: as Google increasingly answers users’ questions directly, referral traffic to the broader web declines substantially, yet these changes do not currently yield commensurate improvements in users’ trust or overall search experience.”

They continue:

“Despite these significant effects on traffic, we find few corresponding gains for users. Removing AI Overviews produced no detectable change in perceived trust in information found on Google, nor on measures of search experience, suggesting that benefits of AI Overviews are not reflected in these outcomes. AI Mode’s effects, relative to current Google Search, decreased perceived trust in information on Google, usefulness, satisfaction, agency and relevance, and increased substitution to competing search engines. Qualitative responses help explain these effects: although some participants appreciated faster access to answers, they more frequently described feeling less in control of their search process, encountered difficulty in reaching desired sites, and saw fewer and less diverse sources. Taken together, these results suggest that in its current form, conversational AI search is viewed less favorably by both publishers and searchers.“

“If users remain firm in their mistrust and dislike of AI Mode,” the researchers write, “Google will face a tension between pushing the product on users (to acclimate them as much as for its other benefits), and risking alienation that could cost it ground to competitors in a search market that today is nearly monopolistic.”

They conclude:

“Given these implications, high stakes for Google, publishers, and users alike, we argue that AI-mediated search should be evaluated not only in terms of the quality of answers generated, but also in terms of how these systems reshape the relationship between users, publishers, and the broader web—from the sustainability of publishers, to Google’s status as an intermediary, to users’ own right to control their discovery, evaluation, and engagement with with information.”

Knowing what we do now, why should SEO/GEO professionals care about “AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence”?

The overarching takeaway from this research supports a widely held view in the community: SEO/GEO is expanding from optimizing for visits to also optimizing for representation within AI-mediated discovery.

Here are five more takeaways:

1. AI visibility and website traffic are becoming separate outcomes.

AI Mode reduced external click-through by 18.8 percentage points, showing that search visibility and referral traffic are becoming less tightly coupled.

2. The search engine is becoming the destination.

Users spent more time inside AI Mode while visiting fewer external sites.

3. Source selection matters more when fewer sources are shown.

Users specifically complained about limited links and source diversity. This begs the question for GEO professionals: if AI interfaces expose users to fewer links and sources, how do brands earn inclusion in that more constrained discovery environment?

4. AI search changes user behavior beyond CTR.

AI Mode reduced search sessions, increased session duration, and pushed more users toward competing search engines. Therefore, our measurement models for SEO/GEO need to consider behavior across the broader discovery journey, not just rankings and organic clicks.

5. Better AI technology does not automatically mean a better search experience.

AI Mode reduced trust, usefulness, satisfaction, agency, and perceived relevance.

In short, this study casts real doubt on claims that AI-driven search can maintain publisher traffic while delivering a better search experience. At least in this experiment, AI Mode kept more attention inside Google while reducing clicks to the open web and leaving users less satisfied, less trusting, and with less sense of control.

Outro

I hope you’ve enjoyed this edition of Hamsterdam Research! 🐹

Feel free to comment below or contact me with your feedback.

Stay tuned for another new article, hopefully next week, or check out related research posts below.

Until next time, enjoy the vibes:

Thanks for reading. Happy optimizing!