Summarizing “What the Author Wrote, What the Agent Reads: Silent Cue Loss in Enterprise Document Ingestion”: A Hamsterdam Research Post

By Ethan Lazuk

Welcome to another edition of Hamsterdam Research! 🐹

This is where we look at recent AI research papers to learn what they’re talking about and explore their hypothetical implications for the future of search and SEO/GEO strategies.

This time, we’ll look at research called, “What the Author Wrote, What the Agent Reads: Silent Cue Loss in Enterprise Document Ingestion.”

It was submitted to arXiv on October 21, 2026, and its authors are Yikai Zhao, Saurabh Pandey, and Pradeep Kumar Misra.

First off, why should SEO/GEO professionals care about this research?

This paper finds that AI systems can successfully extract the words from a document while losing some of the meaning the author intended to communicate. For AI search, this raises an important question of whether just because an AI system can access and process our content does that also mean it understands the content as we intended?

The first takeaway from the research is that content extraction isn’t necessarily the same as content comprehension. The researchers found that document converters could preserve the text of a document while stripping away visual cues that communicated importance, urgency, or other meaning. For SEO/GEO, this means there’s a difference between making content available to an AI assistant and ensuring that the content’s intended meaning survives processing.

Another takeaway was that humans and AI systems may encounter different versions of the same content. A person reading a document can recognize a highlighted exception, red warning, or visually emphasized instruction, whereas an AI agent consuming a converted text representation might receive all the same sentences without knowing which ones were emphasized.

A third takeaway was that AI-generated answers can be confidently wrong even when the underlying source is accurate. The authors found that missing formatting cues didn’t simply cause models to omit information but in some cases, they identified requirements the author hadn’t emphasized. In other words, the problem wasn’t necessarily that the source document contained incorrect information but that the processing pipeline removed information needed to interpret it correctly.

A fourth takeaway was that preserving a signal and explaining what it means are different things. The researchers tested whether attaching semantic labels to formatting cues improved model performance beyond preserving the raw cues alone. They found it did, although the improvement was modest and not consistent across every model. The implication is that AI assistants may benefit not only from receiving a signal, but also from receiving enough context to interpret its meaning.

The final takeaway is that content optimization may need to consider what survives ingestion. Beyond crawling, rendering, and indexing, this paper gives us another layer to think about: what happens when source content is transformed into a representation an AI model can consume? A page or document might look understandable to a human while losing meaningful distinctions during that transformation.

Caveat: This research examines enterprise Word and PDF document ingestion, not Google Search, AI Overviews, or other web-search indexing systems. Thus, we shouldn’t assume those systems exhibit the same failures. That said, this paper gives SEO/GEO professionals a compelling reason to investigate how different AI retrieval and ingestion systems preserve meaning.

Let’s start by reviewing the paper’s abstract.

“Enterprise agents act on documents. Before an agent can follow a policy, answer a compliance question or file a ticket, some ingestion layer must turn a Word file or PDF into tokens – and that layer is where an agent deployed in the wild silently loses information. We show that one layer is lost by every mainstream converter – the meaning an author encodes in how text looks – and that the loss makes agents confidently wrong rather than visibly incomplete. We build a task from 76 real policy documents: recover the requirements the author singled out, scored against the file’s own OOXML properties by a deterministic matcher with no LLM judge. Formatting-blind ingestion collapses from micro F1 0.690 to 0.150 (docling) and 0.075 (plain text), a gap that replicates across six answering models from two vendors – five Claude models spanning the Opus, Sonnet and Haiku tiers, and OpenAI’s GPT-5.6. An agent that cannot see the cue guesses, and docling’s false-positive rate on requirements the author did not flag is four times ours. A span-level leave-one-out localises the loss to two cues – highlight (+0.302) and colour (+0.262), both Holm-adjusted p < 1e-4 – while bold, italic, monospace and strikethrough each sit within +/-0.003 of zero. The same asymmetry appears in PDF. Colour is also the cue no mainstream converter preserves and no parsing benchmark scores, and it is common: 32.8% of 6,460 documents carry a deliberate colour or highlight, 10.3% on an obligation. A pre-registered test shows that typing a cue for the agent adds +0.085 beyond merely preserving it (Holm-adjusted p = 0.049). We validate the labels against rendered page images through an independent modality (kappa = 0.72) and bound what wording alone can achieve (AUC 0.664). For agent builders the practical finding is that ingestion, not reasoning, is where this reliability is won or lost, and that the fix is cheap and measurable.”

Next, let’s break down the paper’s key vocabulary terms:

• Document Ingestion: The process of converting source documents, such as Word files or PDFs, into a format that AI systems can process and interpret.
• Formatting Cues: Visual characteristics of text, such as color, highlighting, bolding, or strikethrough, that communicate additional meaning beyond the words themselves.
• Formatting-Blind Conversion: A document-processing approach that extracts text while discarding visual formatting signals, potentially causing AI systems to lose important context.
• Semantic Role: A label that explicitly identifies the meaning or function of a piece of content, such as CRITICAL_WARNING, WARNING, or DEPRECATED, helping an AI system interpret the author’s intent.
• Cue Typing: The process of assigning meaningful labels to preserved formatting cues rather than simply passing along their visual properties. For example, identifying red text as a critical warning instead of merely recording its color.

Bravo. Let’s take a deeper look at the research paper’s contents now.

If you want to follow along, you can grab a PDF or the HTML version on arXiv.

The paper has five main sections.

We’ll focus on the most relevant ones below.

Introduction

Below I’ll paste in excerpts from the introduction that provide essential context for understanding the paper:

“An enterprise agent deployed in the wild does not read documents; it reads whatever an ingestion layer hands it. Before it can answer a compliance question, follow an operating procedure, or decide whether an action is permitted, some converter has turned a .docx or PDF into tokens. That layer is rarely the object of study, and it is where a large amount of an agent’s practical reliability is decided. …

Enterprise authors mark obligations visually—a red requirement, a highlighted exception, a struck-through value that no longer applies—and every mainstream converter we tested discards that layer while preserving the words around it. An agent reading the result sees a list of requirements with no indication which ones the author flagged, and, as we show, it guesses.

Three properties make this a safety-relevant ingestion problem rather than a formatting inconvenience. First, it is silent: nothing in the pipeline signals that information was dropped, so no monitor fires. Second, it inverts the usual failure mode: the agent does not abstain or return less, it asserts more, and its false-positive rate on requirements the author did not flag rises fourfold. Third, it is common, present in about a third of real documents, so it is not an adversarial edge case but the ordinary state of enterprise corpora.”

Evaluation

I’ll summarize the evaluation section below:

The researchers evaluated whether preserving visual formatting cues during document ingestion improves AI systems’ ability to interpret information accurately. They tested real policy documents, compared different document parsers, evaluated multiple AI models, and validated their findings against an independent benchmark.

Key findings were:

• Visual formatting cues are common in enterprise documents.
• AI systems struggled to identify important information when formatting was removed.
• Color and highlighting were the most consequential formatting cues.
• Explicitly identifying the meaning of formatting improved performance.
• The findings were supported by an independent benchmark.
• The problem persisted across different AI models.
• Similar information loss occurred with PDFs.
• The benefits persisted throughout the document-processing pipeline.
• Missing formatting could lead to incorrect answers without obvious errors.
• Recovering formatting cues involves a trade-off (more token consumption).

The overarching finding was that AI systems can retain the words of a document while losing the signals that explain how those words should be interpreted.

Conclusion

Below are excerpts from the final section:

“An enterprise agent reads whatever its ingestion layer produces, and that layer silently discards the meaning authors encode visually. On 76 real policy documents, recovering the requirements an author singled out falls from micro F1 $0.690$ to $0.150$ (docling) and $0.075$ (plain text), a gap that replicates across six answering models from two vendors. The failure is the difficult kind for autonomy: not an error the pipeline reports, but an agent that answers fluently and wrongly, with four times the false-positive rate on requirements the author never flagged.

The loss is localised. A per-cue leave-one-out attributes essentially all of it to highlight ($+0.302$) and colour ($+0.262$), both $p_{\text{Holm}}<{10}^{-4}$, with bold, italic, monospace and strikethrough each within $\pm 0.003$ of zero—and the same asymmetry reappears in PDF. Colour is simultaneously the cue that matters most and the one no mainstream converter preserves and no parsing benchmark scores. A pre-registered test further shows that typing a cue for the agent adds $+0.085$ beyond preserving it ($p_{\text{Holm}}=0.049$), consistent with evidence that emphasis redirects attention without accuracy reliably following [2].”

Knowing what we do now, why should SEO/GEO professionals care about “What the Author Wrote, What the Agent Reads: Silent Cue Loss in Enterprise Document Ingestion”?

We’ve already discussed how formatting can disappear during ingestion, but the research suggests several more practical implications for content strategy, technical SEO/GEO, measurement, and AI search optimization.

1. Test the machine-readable version of your content, not just the rendered version

For example, take an important PDF, convert it into plain text or Markdown using a document parser, and compare the result with the original. Look for missing relationships, exceptions, emphasis, document versions, and explanatory context.

2. Separate information retrieval accuracy from information interpretation accuracy

As practitioners, we should ask two questions: “Did the AI system retrieve or reference the correct source?” and “Did its answer accurately represent the meaning, qualifications, and importance of the information within that source?”

A brand citation, for example, might be technically correct while the accompanying explanation misrepresents an important product limitation.

3. Write important distinctions so they survive plain-text extraction

Consider a pricing page where a special condition is communicated only through a colored box or highlighted footnote. A person might immediately understand that the condition changes the offer, but an ingestion system that discards the formatting might not. So as a practical takeaway, make business-critical distinctions explicit in the wording itself, rather than relying exclusively on visual presentation.

4. Monitor false claims, not just missing mentions

GEO measurement often concentrates on whether a brand is mentioned, cited, or included in an AI-generated recommendation, but this study shows why false positives deserve attention, too.

Look for false claims and take corrective actions, such as:

• Attribute capabilities to products that they don’t actually have.
• Present optional features as standard inclusions.
• Turn conditional recommendations into universal ones.
• Omit restrictions that materially change the answer.

Outro

This research gives us three different levels of content fidelity to think about:

1. Textual fidelity: Did the AI system receive the correct words?
2. Semantic fidelity: Did the relationships, qualifications, and intended significance of those words survive ingestion?
3. Answer fidelity: Did the AI-generated response accurately communicate that information to the user?

This three-level framework provides a useful way to think about SEO/GEO because each level presents a different opportunity for information to be misunderstood. A document can pass the first test while failing the second, and a system can retrieve relevant content while still producing a misleading answer.

As SEO/GEO professionals, we may succeed in making content discoverable and retrievable, yet still need to evaluate whether AI systems accurately represent what the content says.

I hope you’ve enjoyed this edition of Hamsterdam Research! 🐹

Feel free to comment below or contact me with your feedback.

Stay tuned for another new article, hopefully next week, or check out related research posts below.

Until next time, enjoy the vibes:

Thanks for reading. Happy optimizing!