What Does Your GEO Visibility Score Actually Measure? Why Prompt Selection Matters (Hamsterdam Research)

By Ethan Lazuk

Welcome to another edition of Hamsterdam Research! 🐹

This is where we look at recent AI research papers to learn what they’re talking about and explore their hypothetical implications for the future of search and SEO/GEO strategies.

This time, we’ll look at research called, “Measuring GEO Visibility: Prompt Corpora Define the Answer Market.”

It was submitted to arXiv on September 6, 2026, and its author is Olivier Martinez.

In short, why should SEO/GEO professionals care about this research?

Essentially, this paper is a critical survey and methodological analysis that shows that the prompts we choose to track as SEO/GEO professionals help determine the visibility we report, but those prompts may not represent actual audience demand.

The practical implications of this research are that:

• A higher score doesn’t necessarily mean better performance: Changing the prompt mix or how much weight each category receives can change the overall score.
• Comparisons need a common basis: Scores across tools, competitors, or time periods can mislead if the measurement methods differ.
• Citations don’t prove influence: Being cited doesn’t establish that your content shaped the answer, earned someone’s trust, or drove a purchase.
• Strategy depends on meaningful measurement: Otherwise, you could optimize for a selected set of test prompts without knowing whether you’re improving visibility in conversations that matter to your customers.

The paper’s relevance is primarily about whether we can trust and correctly interpret GEO reporting, rather than how to earn more AI mentions.

Let’s start by reviewing the paper’s abstract.

Here’s the abstract (with my highlights):

“To measure the visibility of a source or brand in a generative engine’s answers, a set of prompts is submitted to the engine, and displayed sources, citations, or mentions are recorded according to specified rules. These observations are aggregated into a GEO (generative engine optimization) visibility score. The choice of prompts determines which situations are evaluated; the weights determine their relative importance in the calculation. We call this weighted set of answer opportunities an “answer market.” This constructed market does not necessarily represent actual user demand.

Prompt wording can alter document retrieval, the sources brought into competition, and the answers produced. Once these answers have been generated, calculating the visibility score requires identifying the source appearances, citations, or mentions of interest. If this task is assigned to a language model, a second instruction enters the process: the instruction telling the model how to evaluate the answers. This instruction can change the scoring even when the evaluated answers remain identical. Our critical survey examines how these choices help define what a GEO score measures. It draws on research that examines whether the indicators used measure the intended phenomenon, as well as research on total survey error and information retrieval evaluation. The proposed framework makes explicit the evaluated situations and their annotation, prompt formulations, execution conditions, weights, and scoring rules.

Two calculations illustrate the consequences of these choices. Using published aggregate rates from three selected subcorpora in Grossman et al., two weighting schemes yield generative-answer activation rates of 39.7% and 70.5%, without changing the rates within each subcorpus. These results concern whether a generative answer appears; they do not measure brand visibility. A fictional example shows that reweighting can reverse the ranking of two systems when each performs better in different categories of situations.

When weights are unknown or remain to be chosen, the framework proposes reporting a set of admissible scores. It distinguishes two approaches: determining the values compatible with the data and assumptions about a target population, which concerns partial identification; and examining how the score varies across weighting conventions, which concerns normative sensitivity.

The presence of a citation is insufficient to establish a source’s contribution to an answer. To examine this contribution, the article defines a comparison between answers produced with and without a source in a controlled documentary context. It distinguishes this contrast from an intervention on the full engine in the presence of competing sources. Together, these contributions provide a framework for interpreting GEO scores, supported by reproducible calculations. No new experiments are reported; the general empirical validity of the proposed protocol remains to be assessed.”

Next, let’s break down the paper’s key vocabulary terms:

• GEO visibility score: A measure summarizing how often (or according to specified scoring rules) a brand or source appears in AI answers under a particular testing setup.
• Prompt corpus: The collection of prompts used to test visibility. Its composition determines which questions and situations the measurement covers.
• Answer market: The weighted set of opportunities for a brand or source to appear in the answers being evaluated. This constructed market does not necessarily represent actual user demand.
• Weights: The relative importance assigned to different prompts or categories when calculating an overall score. Changing them can change the result without changing any individual answer.
• Construct validity: Whether a measurement captures the concept it claims to measure, for example, whether a visibility score supports claims about visibility among your intended audience.
• Partial identification: Determining a range or set of values compatible with available data and explicit assumptions when the evidence cannot establish one exact value.
• Normative sensitivity: How a score changes under different choices about what should receive more or less weight, such as prioritizing certain customer situations.
• Source attribution: Assessing a source’s contribution to an answer. The paper distinguishes appearing in a citation from actually influencing the answer.
• Source-removal contrast: Comparing answers generated with and without a source in a controlled context to investigate its contribution.
• LLM judge: A language model used to evaluate or score answers. Its instructions can change the scores even when the answers being evaluated stay identical.

Great. Let’s take a deeper look at the research paper’s contents now.

If you want to follow along, you can grab a PDF or the HTML version on arXiv.

The paper has 10 main sections.

We’ll focus on the most relevant ones below.

Introduction

The introduction argues that a GEO visibility score only makes sense when you know how it was measured. The prompts selected, their relative weights, and the scoring rules define the “answer market” being tested.

As the author cautions, “This constructed market need not represent human demand.” A source appearing in 40% of answers means little without knowing what questions were asked and under what conditions.

Prompts also actively affect the measurement as they can influence whether search happens, which documents are retrieved, and what the answer contains. If an AI model scores those answers, its judging instructions can change the scores even when the answers stay the same. That is why the paper calls the testing setup an “active measurement instrument.”

The paper also proposes ways to document these choices, examine how weighting affects scores, and investigate a source’s contribution by removing it in controlled comparisons. These are methodological proposals supported by examples and existing data, but their practical usefulness still needs empirical evaluation.

Discussion and Limitations

Note: I don’t typically jump this far ahead when reviewing a research paper, but I believe this section summarizes some of the others we have skipped. (Feel free to check out the PDF for the full paper.) That said, there is one section worth noting: Section 7.5, “A Secondary Demonstrator Using Published Aggregates.”

How weighting changes the reported result:

• Two weighting schemes applied to published data produced overall generative-answer activation rates of 39.7% and 70.5%.
• The rates within each subgroup stayed the same but their relative importance in the calculation changed.
• These figures measure whether a generative answer appeared, not whether a particular brand appeared.

The discussion argues that prompts help create the conditions in which visibility is measured. An open-ended recommendation prompt and a comparison prompt naming two brands create different opportunities for those brands to appear. For example, compare tracking “What are the best SEO tools?” with “How does Ahrefs compare with Semrush?” The second prompt supplies the brands being evaluated, which changes the opportunity for those brands to appear.

Prompts can also change what the engine searches for, which documents it retrieves, and which sources compete.

Several practical points follow from this section:

• Controlled tests don’t automatically describe real-world engines: Giving an AI a fixed set of documents tests its behavior within that context. Letting it find sources also tests retrieval and document selection.
• Answer changes and scoring changes are different: An AI judge can score an unchanged answer differently after its instructions change. Researchers should document prompt selection, answer generation, and scoring separately.
• Rankings depend on the comparison conditions: A ranking is robust when it holds across the justified weighting choices being considered, with the same weights applied to every system. Otherwise, conclusions should explain which choices affect the ordering.
• The framework still needs validation: The examples demonstrate mathematical possibilities and the effects of reweighting existing data. They do not establish how prompts affect a real population of users.
• The evidence has limits: The survey is selective, largely English-language, and constrained by missing data. Poorly justified assumptions can make findings appear more stable (or less conclusive) than warranted.

Lastly, visibility is not a direct measure of marketing impact. Mentions, citations, and exposure alone do not establish attention, trust, persuasion, or purchases. Those outcomes require research involving people and their behavior.

Conclusion

“A GEO visibility score summarizes the observed presence of sources or brands in answers produced under a given protocol. The prompt corpus and its weights delimit the answer market over which this visibility is evaluated. Prompts play a dual role: they select the situations covered and elicit the system’s behavior, potentially altering retrieval and the sources placed in competition. When an LLM judge is used, its instruction adds another dependence on formulation. This connection gives meaning to the article’s central distinction: an influence on the output is a behavioral fact; participation in the definition of measurement is a property of the evaluation design.

The developments presented here clarify the consequences of this distinction. The analytical example establishes that changing weights can reverse a ranking when system profiles cross between strata. The demonstrator based on published aggregates shows how panel composition and denominator conventions change the quantities reported. It concerns the activation of generative answers; it is not an observed measure of a publisher’s visibility. These results explain why a difference between two scores must be examined in light of the conditions under which each was constructed.“

Knowing what we do now, why should SEO/GEO professionals care about “Measuring GEO Visibility: Prompt Corpora Define the Answer Market”?

SEO/GEO professionals should care about this research because it shows that the way we measure AI visibility can shape the conclusions we draw and the strategies we invest in. This paper gives us a framework for questioning what our reporting actually represents.

• Are you tracking your audience’s questions? A chosen prompt set defines the opportunities being measured, but it doesn’t automatically represent real customer demand.
• Does a score increase reflect progress? Changes in prompt selection, weighting, or scoring can move the number without demonstrating improved visibility under comparable conditions.
• Are comparisons meaningful? Evaluating competitors, tools, or performance over time requires consistent methods and a clearly defined scope.
• Will a controlled finding apply to live search? A tactic tested with supplied documents may behave differently when an engine chooses which sources to retrieve.
• What business outcome does visibility support? A mention or citation alone doesn’t demonstrate influence, trust, or sales.

The practical value of this research is better judgment about GEO metrics — knowing what a score supports, what remains uncertain, and what additional evidence you need before acting on it.

In terms of actionable takeaways, we should document our prompt set, weights, engine, collection dates, and scoring rules before we interpret changes in AI visibility scores over time.

Note: The paper proposes a measurement framework, but it does not validate a particular optimization tactic.

Outro

I hope you’ve enjoyed this edition of Hamsterdam Research! 🐹

Feel free to comment below or contact me with your feedback.

Stay tuned for another new article, hopefully next week, or check out related research posts below.

Until next time, enjoy the vibes:

Thanks for reading. Happy optimizing!