LLM◆ AI-generated · Sourced
On-Device Deep Research at 4B: Exposure Bounds Faithfulness, Retrieval Bounds Coverage
TL;DR
arXiv:2607.12257v1 shows that for a deployable 4B-parameter LLM running on a 24 GB laptop, cited claim faithfulness is bounded by per-source exposure length (400 vs. 1500 characters), lifting from 0.37–0.45 to 0.58 and converging across gold and retrieved sources; trustworthy coverage, however, depends critically on retrieval quality—not exposure—and remains uncaptured by increasing input context alone.
Core Contribution
- This work is the first to decouple citation evaluation for on-device LLMs into two orthogonal metrics: cited claim faithfulness (whether the claim is supported by the cited source) and trustworthy coverage (whether the agent cites the right sources — i.e., authoritative, relevant, and comprehensive ones).
Exposure Length Governs Faithfulness
- The study fixes a single 4B generator (no fine-tuning) on a 24 GB laptop and varies per-source exposure: 400 vs. 1500 characters.
- It crosses this with source quality: gold papers (manually curated high-quality research papers) vs. retrieved papers (outputs of a realistic RAG pipeline).
- Key results:
- On retrieved sources, faithfulness rises from 0.45 → 0.58;
- On gold sources, faithfulness rises from 0.37 → 0.58;
- Both converge precisely at 1500 characters — proving faithfulness is exposure-bound, not source-correctness-bound.
- The exposure gain is robust under a second independent judge; exact convergence holds strictly under the primary judge and approximately under the secondary.
Retrieval Quality Determines Trustworthy Coverage
- Trustworthy coverage does not improve with longer exposure — it hinges on the quality of the retrieved corpus itself.
- With gold papers, the model activates key arguments more completely and links across sources meaningfully; with retrieved papers, noise, redundancy, and bias persist even at 1500 chars, causing omission of critical supporting literature.
- This reveals a clean functional division in RAG pipelines: retrieval owns coverage breadth and source authority; reading + generation owns faithful grounding — neither can compensate for the other’s failure.
Sources (compliance trail)
https://arxiv.org/abs/2607.12257