Summarizing “Learning Multi-Step Query Rewriting via Corpus Feedback for Conversational Search”: A Hamsterdam Research Post

By Ethan Lazuk

Welcome to another edition of Hamsterdam Research! 🐹

This is where we look at recent AI research papers to learn what they’re talking about and explore their hypothetical implications for the future of search and SEO/GEO strategies.

This time, we’ll look at research called, “Learning Multi-Step Query Rewriting via Corpus Feedback for Conversational Search.”

It was submitted to arXiv on October 7, 2026, and its authors are João Coelho, Hong Wang, Jie Yuan, Zhuoer Wang, Samson Koelle, and Wei Niu.

First off, why should SEO/GEO professionals care about this research?

The main takeaway from this research is that AI search systems can improve retrieval by searching, examining the results, and then rewriting the queries using what they’ve learned. For SEO/GEO professionals, this raises the possibility that content visibility may depend not only on how well a page matches an initial query, but also on how well it matches the increasingly specific queries an AI search agent develops during its research.

Whereas traditional conversational query rewriting generally involves turning a user’s context-dependent question into a standalone search query and retrieving documents once, this research proposes an approach where an AI agent searches, examines the retrieved passages, and uses that evidence to decide how to search again.

The agent in this research can perform three rewriting operations:

1. Intent resolution: Clarify what the user is actually asking, using the conversation history.
2. Lexical reformulation: Adjust the query’s wording to better match the terminology found in relevant documents.
3. Pseudo-document generation: Generate a hypothetical answer passage informed by earlier retrieval results, then use that passage to find better matches.

In experiments using conversational search benchmarks, the multi-step approach outperformed several existing query-rewriting methods on ranking precision in this research.

This research has three practical implications for AI search optimization:

1. The vocabulary within your content matters beyond the initial search.
2. Topical depth may help content remain relevant as queries evolve.
3. Retrieval visibility may be a multi-step problem.

In summary, the larger takeaway is that optimizing content for AI search may increasingly mean thinking beyond the words users type and considering the terminology, concepts, and relationships an AI agent could discover and pursue throughout a search session.

Let’s start by reviewing the paper’s abstract.

“Conversational Query Rewriting (CQR) turns a context dependent user turn into a standalone query for a retriever, and most methods do this in a single step from the dialogue history before retrieving once. The rewrite is therefore fixed before any corpus evidence is available to correct its reference resolution or its vocabulary. We recast CQR as a sequential retrieval problem: an agent rewrites the current turn, retrieves, and conditions its next rewrite on the returned passages. The agent acts in a typed space of three rewriting operations, resolving conversational intent into a standalone query, generating lexical reformulations, or synthesizing pseudo-documents for document-to-document matching, together with a stop action that ends the episode. We train the policy with supervised fine-tuning followed by reinforcement learning against a single retrieval-quality reward, using no human rewrite annotations. Across TopiOCQA and QReCC, the agent outperforms several retrieval-aligned baselines, while remaining effective across retrieval backends and generalizing to the CAsT benchmarks without additional training. Further analysis shows that, through retrieval-reward optimization alone, the learned policy develops a behavior of grounding pseudo-documents in passages retrieved by earlier steps, substantially improving retrieval.”

Next, let’s break down the paper’s key vocabulary terms:

• Conversational Query Rewriting (CQR): The process of transforming a question that depends on previous conversation context into a standalone search query that a retrieval system can understand.
• Corpus Feedback: Information gathered from previously retrieved documents that an AI agent uses to improve subsequent search queries, including adopting relevant terminology and concepts.
• Intent Resolution: The process of identifying what a user actually wants to know by interpreting their question in the context of earlier conversation turns.
• Lexical Reformulation: Rewriting a search query using alternative words, synonyms, or more precise terminology to better match relevant documents.
• Pseudo-Document Generation: Creating a hypothetical passage that could answer a user’s question, then using that passage as a search query to retrieve documents with similar content.
• HyDE (Hypothetical Document Embeddings): A retrieval technique that generates a hypothetical answer document and uses its representation to search for relevant real documents. In this paper, the agent’s HyDE-style operation can incorporate information from earlier retrieval steps.
• Pseudo-Relevance Feedback (PRF): A traditional information retrieval technique that assumes top-ranked search results are potentially relevant and uses information from them to refine subsequent searches.
• Reinforcement Learning (RL): A machine learning approach in which an agent learns which actions produce better outcomes. In this study, the agent learns query-rewriting strategies through rewards based on retrieval quality.
• Rank-Incentive Reward Shaping (RIRS): A reward method that assigns scores based on where a relevant passage appears in search results, encouraging the agent to move relevant documents higher in the rankings.
• nDCG@3 (Normalized Discounted Cumulative Gain at 3): A retrieval evaluation metric that measures how effectively a search system ranks relevant documents within its top three results, giving greater weight to documents appearing higher in the ranking.

Esta bien. Let’s take a deeper look at the research paper’s contents now.

If you want to follow along, you can grab a PDF or the HTML version on arXiv.

The paper has 5 main sections.

We’ll focus on the most relevant ones below.

Introduction

“Conversational search systems must handle queries whose meaning depends on prior turns,” explain the authors. “Conversational Query Rewriting (CQR) transforms the current turn into a self-contained query that an existing retriever can process, and it is the standard interface between conversational systems and retrieval infrastructure.”

“Most CQR methods produce a single rewrite from the conversational history and issue it directly to the retriever,” they explain. “Consequently, these methods cannot adapt the query to the language and entities used in the target corpus.”

“Using retrieval results to inform a subsequent query is, however, a long-standing idea in information retrieval.” Yet, “How a conversational query should be reformulated after observing retrieved evidence, and how successive retrievals can progressively refine it, remains comparatively unexplored” relative to agentic search systems that use reasoning.

Next they explain their work:

“In this work, we formulate conversational query rewriting as a sequential retrieval problem. An agent repeatedly reformulates the current turn, issues each reformulation to the retriever, and uses the returned passages as feedback for its next decision. It chooses among three typed rewriting operations: (i) resolving conversational intent into a self-contained query, (ii) generating lexical reformulations, and (iii) synthesizing pseudo-documents for document-to-document matching [9]; it can also issue a stop action that ends the episode. We train the policy with supervised fine-tuning followed by reinforcement learning, using retrieval quality as the only objective and no human rewrite annotations.”

Across the settings they tested, “the multi-step agent improves over multiple baselines, with further analysis showing that retrieval-reward optimization alone leads the learned policy to progressively ground its reformulations in corpus evidence: it first resolves the query intent, optionally refines its lexical formulation, and finally generates pseudo-documents conditioned on passages retrieved by earlier steps.”

Related Work

Related work breaks down into three categories:

1. Conversational Query Rewriting

Earlier approaches rewrote conversational questions into standalone search queries, often using human examples or reinforcement learning to improve retrieval. This research goes further by allowing the agent to examine search results and use that evidence to refine subsequent queries.

2. Pseudo-Relevance Feedback

Traditional methods improve search queries using terms or information from initially retrieved documents, but typically follow predefined rules for doing so. This research allows an AI agent to learn which rewriting strategy to use, incorporate evidence from earlier results, and decide when to stop searching.

3. Reinforcement Learning for Search Agents

Previous research primarily trained AI search agents to produce better final answers, rather than focusing specifically on retrieval quality. This study instead rewards the agent for ranking relevant documents higher, teaching it to develop effective multi-step search strategies without requiring reference answers or human-written query rewrites.

Conclusion

“In this paper we studied conversational query rewriting as a sequential retrieval process driven by feedback from intermediate search results. We trained a multi-step agent with reinforcement learning and no human rewrite annotations. Rewarded solely on retrieval quality, the policy converges to a consistent action order: it first resolves conversational intent, optionally refines the lexical formulation, and then generates pseudo-documents grounded in the passages retrieved by earlier steps. This formulation achieves the strongest ranking precision among the compared methods on TopiOCQA and QReCC, transfers zero-shot to TREC CAsT, and remains effective under dense retrievers.”

I also generated my own diagram to explain what the authors achieved with their research, to help visualize the process:

Knowing what we do now, why should SEO/GEO professionals care about “Learning Multi-Step Query Rewriting via Corpus Feedback for Conversational Search”?

The biggest takeaway from this research is that AI search agents don’t necessarily rely on a single search query to find relevant information, but rather they can learn from initial search results, refine their queries, and retrieve better documents with each step.

For SEO/GEO professionals, this raises the consideration that optimizing for the user’s original question may not be enough, as content should also be relevant to the terminology, entities, and follow-up searches an AI agent might encounter as it investigates a topic.

Here are several practical SEO/GEO strategies inspired by this research:

1. Optimize for query variations, not just individual keywords

The researchers found that their agent learned to reformulate queries using terminology discovered in retrieved passages. For example, a page about measuring AI search visibility might also discuss AI citation tracking, brand mention frequency, and share of voice in LLM responses.

2. Build content around related questions and follow-up searches

Conversational search often involves questions that depend on earlier exchanges. The researchers specifically studied how an agent resolves these references before retrieving information. For example, an article about measuring AI visibility could include sections on tracking citations, monitoring brand mentions, interpreting fluctuations, and comparing AI search platforms.

3. Use precise terminology and clearly explain important entities

One of the study’s most interesting findings is that the agent learned to incorporate vocabulary from previously retrieved documents into later query rewrites. For example, instead of simply saying that a tool measures AI visibility, explain that it monitors brand mentions across specific AI platforms, tracks prompt-level results, and evaluates changes in citation frequency.

4. Develop comprehensive topical resources

The study suggests that retrieval improves when an agent can refine its information need using evidence discovered during earlier searches. This creates a potential advantage for content that thoroughly addresses a topic and its meaningful subtopics.

5. Make individual passages useful and understandable

The research evaluates the retrieval of relevant passages, rather than entire pages as indivisible units.

6. Expand GEO testing beyond single-turn prompts

The agent achieved better retrieval rankings by conducting multiple search and rewriting steps. Simply testing a single prompt may therefore miss how content could become relevant during a longer information-seeking process.

7. Prioritize useful evidence over repeated keyword targeting

The agent’s strongest gains came when it generated hypothetical answer passages grounded in previously retrieved evidence, rather than relying on unsupported generation. The relevant lesson isn’t that every page needs original research. It’s that substantive information gives retrieval systems more meaningful evidence to work with.

Outro

While traditionally SEO has emphasized matching content to the queries users enter, this research encourages a broader perspective where we consider how an AI search agent’s information needs and search vocabulary might evolve after it encounters your content and other sources.

Practially speaking, that means producing content that answers the initial question, addresses likely follow-up questions, uses precise terminology, and provides useful evidence that could support deeper investigation.

The most actionable principle from this research is to optimize for the entire information-seeking process, not just the starting query. As conversational search develops, understanding how content can be discovered during intermediate retrieval steps may become an increasingly valuable part of SEO/GEO strategy.

I hope you’ve enjoyed this edition of Hamsterdam Research! 🐹

Feel free to comment below or contact me with your feedback.

Stay tuned for another new article, hopefully next week, or check out related research posts below.

Until next time, enjoy the vibes:

Thanks for reading. Happy optimizing!