What is the cherry-picking of information from RAG systems like Perplexity and Copilot?

SEO for AI search isn’t only about getting mentions and citations, but also ensuring those mentions fit the context of the original article.

I recently read an interesting article from Unite.AI called “New Research Finds Sixteen Major Problems With RAG Systems, Including Perplexity.”

The article references research that found causes for concern with RAG systems including You Chat, Bing Copilot, and Perplexity.

Some of the causes for concern included overly confident language, misattributing sources, and reinforcing perceived user bias.

But one of the sixteen stood out to me for further investigation: cherry-picking information from inferred context.

The reason this stood is because it’s where the RAG agent seeks answers to support an estimation of what the user wants to hear.

As SEOs, this is interesting because it could influence how information (and citations) get perceived when included in AI answers.

I decided to go to the research paper, “Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responses,” to dig into the section on cherry-picking information.

The authors write:

“When participants posed expert or opinion-based questions, they noticed that the system often selectively presented information from retrieved sources, highlighting a particular perspective instead of a comprehensive view of the article.”

In other words, say that an article has pros and cons on a topic, the participants found the model was more likely to site whichever side reinforces the user’s intention and “does not accurately represent the full scope of the source material.”

Why does this matter?

It could mean AI search misrepresents a source by “taking statements out of context to fit the query.”

As SEOs, were often focused on getting included in AI answers, but we may not spend as much time on whether those inclusions accurately reflect the content’s context.

As the researchers point out, this could potentially distort the original intent of the content.

This is a reminder that AI search optimization isn’t only about getting your content mentioned and cited in answers but also ensuring the context fits your intentions.

This could influence content strategy and how topics are chosen based on a buyer’s journey.

Thanks for reading. Happy marketing! 🤗

Related posts