Evaluating “The Fellowship of the Query: Learning Retrieval Actions” for SEO/GEO Insights: A Hamsterdam Research Post

By Ethan Lazuk

Welcome to another edition of Hamsterdam Research! 🐹

This is where we look at recent AI research papers to learn what they’re talking about and explore their hypothetical implications for the future of search and SEO/GEO strategies.

This time, we’ll look at research called, “The Fellowship of the Query: Learning Retrieval Actions.”

It was submitted to arXiv on September 23, 2026, and its authors are Mohammed Al-Maamari, Saber Zerhoudi, Michael Granitzer, and Jelena Mitrović.

Firstly, why should SEO/GEO professionals care about this research?

This research can change our mental model for what “retrieval” in an AI system can look like. The key idea is that retrieval doesn’t have to be one query leads to one search which leads to one set of results. Rather, in the paper’s system, retrieval is treated as a sequence of a decisions — the model can decompose the question, search, reformulate the query, extract evidence, synthesize facts, verify its progress, and then decide when it has enough information to finish the retrieval stage.

That retrieval is a sequence of decisions matters for SEO/GEO for several reasons:

• Visibility can potentially happen across multiple retrieval steps. A page may not be relevant to the original wording of the user’s question but could become relevant after the system decomposes or reformulates it.
• Being retrieved may not be enough. The system still has to decide what evidence to extract and carry forward into its accumulated state.
• Content can play a role beyond simply ranking for a query. In this kind of system, information from a page can be extracted, combined with other evidence, and used to help the system decide what to search for or verify next.
• The system can learn how to move through that search process. The researchers trained a small model to choose among actions like searching, reformulating, extracting evidence, verifying, and finishing, and fine-tuning made it much better at selecting the next step.

In short, AI search may be better understood as a retrieval process rather than a retrieval event.

Let’s start by reviewing the paper’s abstract.

Here’s the abstract in full:

“Retrieval-augmented question answering requires control decisions about when to decompose a question, search, reformulate, extract evidence, synthesize facts, verify progress, and stop. We study whether trajectory fine-tuning can improve small language models (SLMs) as next-action controllers. We additionally evaluate a low-resource setting in which a single SLM serves as both the controller and the final-answer generator. From accepted teacher search traces, we build a seven-way action-prediction task, where the model predicts the next structured teacher action from the current trajectory state, and evaluate LoRA-supervised fine-tuning across SLMs and xSLMs as controllers. On 1,646 held-out action examples, Granite 4.1 3B trained on 13,194 actions reaches macro-F1 0.6536, compared with 0.1736 for zero-shot prompting of the same model and 0.5399 for a TF-IDF logistic-regression baseline. In an end-to-end controller/generator swap evaluation over 149 held-out trajectories, using the fine-tuned model for both roles improves Exact Match from 0.7530 to 0.7946 and token F1 from 0.7783 to 0.8295 compared with using the base model as both controller and generator. The cross-role conditions show that the fine-tuned controller increases evidence-fact recording when the generator is fixed, while controller-only final-answer gains are not statistically clear. Overall, trajectory supervision improves action prediction and evidence-recording behaviour in this evaluated pipeline. The code used in this paper is publicly available.¹“

To break it down a little bit, query fanout, a familiar term for SEO/GEOs, is about taking one user question and expanding it into multiple related searches or subqueries. This paper includes that behavior through decompose and reformulate, but then it asks what the system should do after and between those searches. The seven actions are: decompose, search, reformulate, extract, synthesize, verify, and finish.

Next, let’s break down the paper’s key vocabulary terms:

• Decompose: Break a complex question into smaller sub-questions or tasks.
• Search: Run a search to retrieve information that may help answer the question.
• Reformulate: Rewrite or adjust the query when the current search is not producing the information needed.
• Extract: Pull useful facts or evidence from retrieved documents.
• Synthesize: Combine information from different pieces of evidence into a more complete understanding.
• Verify: Check whether the system has enough reliable information or whether more searching is needed.
• Finish: Stop the search process and move on to generating the final answer.
• Search control: The process of deciding which of those actions the system should take next.
• Trajectory: The full sequence of searches, actions, evidence, and intermediate steps the system takes before reaching an answer.
• Next-action controller: The model responsible for choosing the next step in that search trajectory.

Esta bien. Let’s take a deeper look at the research paper’s contents now.

If you want to follow along, you can grab a PDF or the HTML version on arXiv.

The paper has six main sections.

We’ll focus on the most relevant ones below.

Introduction

I’ll pull out excerpts from the introduction to help summarize it:

“In multi-hop question answering, retrieval is not a single operation. A system must decide when to decompose a question, search, reformulate a query, extract evidence, synthesize intermediate facts, verify progress, and produce a final answer. This creates a search-control problem: final answer quality depends on both generation and the action sequence that constructs the evidence state. …

We study whether lightweight LoRA supervision enables xSLMs and SLMs to imitate a structured seven-action search-control protocol, and whether this changes evidence-fact recording in a constrained multi-hop retrieval pipeline. The accepted teacher trajectories lead to correct answers under our filtering protocol, but we do not assume that every action in those trajectories is optimal; another trajectory could reach the same answer with different or fewer actions. …

This motivation is consistent with prior models that describe search as an evolving and staged process rather than a single query-response event. …

This work contributes a focused evaluation of search-flow control for RAG-style question answering as supervised action prediction from structured teacher trajectories.”

Experiments

To summarize their experiments, the researchers essentially showed that:

1. Small language models can be trained to manage a multi-step retrieval process
2. Such models can learn much of that behavior from a relatively modest amount of training data.
3. Better retrieval control can lead to substantially better evidence gathering and modest improvements in final answers.

Discussion

The discussion section makes four main points:

1. The seven-action retrieval process can be learned: Fine-tuned small models were much better at choosing the next action than zero-shot models.
2. Some actions are harder than others: Class imbalance affected performance, especially for the rare verify action.
3. Better retrieval control improves evidence gathering: The fine-tuned controller recorded more evidence-backed intermediate facts, although controller-only gains in final answer quality were modest.
4. There are trade-offs and limitations: Fine-tuning hurt performance on unrelated benchmarks, and the experiments used a small, controlled retrieval environment rather than open-web search, so the results should not be generalized too broadly.

In summary, the discussion section concludes that fine-tuning can teach small models to make better retrieval decisions and gather more useful evidence, but the gains in final answer quality are modest and the results come with important limitations.

Conclusion

“We evaluated trajectory fine-tuning for search-control actions in SLMs and xSLMs. On an acceptance-filtered held-out 7-way action-prediction task, Granite 4.1 3B LoRA SFT trained on the full 13,194-sample training split reaches macro-F1 0.6536, improving over zero-shot prompting and a TF-IDF baseline. In the reported training-size sweep, the 3,000-training-sample setting obtains macro-F1 0.6489, close to the best observed value of 0.6625, and a single multiclass controller is stronger in aggregate than independent binary action controllers for the evaluated Qwen 3.5 0.8B setting. In the end-to-end controller/generator swap, using the fine-tuned Granite 4.1 3B model for both roles improves token F1 relative to base/base, and replacing only the controller substantially increases evidence-fact recording under a fixed generator. The evidence supports a focused conclusion: trajectory supervision can improve action prediction and evidence-fact recording behaviour for an SLM in this evaluated pipeline, while unfiltered-query evaluation and robust final-answer generation remain open issues.”

To summarize, the researchers found that small language models can be trained to make better decisions about what to do next during a multi-step search process. They also found that a few thousand training examples were enough to get close to the best performance, and that one model choosing among all seven actions worked better overall than using separate models for each action.

When the fine-tuned model handled both search control and answer generation, the final answers improved, and a better controller helped the system capture more useful evidence along the way.

The main caveat is that this was tested in a controlled setup, so it is still unclear how well the approach would work on unrestricted queries and more realistic search environments.

Knowing what we do now, why should SEO/GEO professionals care about “The Fellowship of the Query: Learning Retrieval Actions”?

This paper offers a better mental model for how AI retrieval can work, showing that search isn’t necessarily a single retrieval event but rather it can be a sequence of decisions.

The system in the paper can decompose, search, reformulate, extract, synthesize, verify, and finish, meaning the original user query may only be the starting point. The system can branch into subquestions, rewrite searches, gather evidence, combine what it finds, and decide whether it needs to keep searching.

For SEO/GEO, the implications of this research matter because:

• You are not only optimizing for the original query: Content may become relevant during a later decomposition or reformulated search.
• Ranking is only part of the story: A page can be retrieved, but the system still has to recognize useful information, extract it, and carry it forward as evidence.
• Clear, extractable facts may matter more in AI retrieval: The paper shows that better search control substantially increased the amount of evidence-backed information the system recorded before answering.
• AI systems can learn how to manage search itself: The researchers were able to fine-tune small models to become much better at deciding which retrieval action should happen next.
• Query fanout may be only one part of a larger process: Decomposing and reformulating queries resemble the fanout concepts SEO/GEO professionals already discuss, but this paper goes further by showing a system deciding what to do with the retrieved information afterward.

Outro

The practical GEO takeaway from this research isn’t “How do I rank for this exact prompt?” but rather “How can my content remain useful across the different searches, subquestions, and evidence-gathering steps an AI system might take before producing an answer?”

I hope you’ve enjoyed this edition of Hamsterdam Research! 🐹

Feel free to comment below or contact me with your feedback.

Stay tuned for another new article, hopefully next week, or check out related research posts below.

Until next time, enjoy the vibes:

Thanks for reading. Happy optimizing!