Paper quantifies ‘clinical lost-in-the-middle’ effect in EHR processing and proposes query-conditioned context selection

Electronic health records increasingly exceed 100,000 tokens per patient, but large language models suffer from the lost-in-the-middle (LitM) effect: information near the center of long contexts is retrieved less reliably than information near the edges. In clinical settings, this can mean the most consequential fact in a patient note sits at the center, where models perform worst. The authors term this the clinical lost-in-the-middle (CLitM) problem and provide its first systematic characterization using the MedAlign dataset.

Across 2,196 instruction-response pairs and six language models, the study finds a 21.9 percentage-point gap between peak accuracy (59.5%, 95% CI [46.3, 71.0]) in the 20–30% decile of the EHR timeline and trough accuracy (37.6%, 95% CI [23.2, 52.5]) in the 70–80% decile. Notably, 67.8% of reference answers fall between the 10th and 90th percentiles of the EHR timeline—inside the CLitM trough—indicating that the most critical evidence often coincides with the model’s weakest performance.

The authors introduce Query-Conditioned Clinical Suppression (QCCS), a lightweight, query-conditioned selection gate designed to mitigate CLitM. In evaluations with Qwen2.5-7B-Instruct (16k context), QCCS outperforms five comparators—BM25, BM25 with section-header filtering, dense retrieval, cross-encoder reranking, and full-context prompting—under LLM-as-judge scoring. On middle-position instructions, QCCS reaches 16.7% accuracy versus 3.3% for BM25, 0.0% for cross-encoder and dense retrieval, and 6.7% for full context. Overall, QCCS achieves 25.3% accuracy versus at most 3.6% for retrieval-only comparators.

The advantage of QCCS is not explained by retrieval recall alone. At k=20, BM25 retrieves the gold evidence sentence in 98.8% of instructions (QCCS: 34.9%), yet retrieval baselines achieve at most 2.6% accuracy even when they retrieve the correct evidence. In contrast, QCCS reaches 25.0% accuracy even when it does not retrieve the gold sentence, indicating that query-aligned context selection better predicts instruction-following accuracy than gold-sentence retrieval recall.

The paper is a proof-of-concept evaluation and includes code and data links. It highlights the need for long-context methods that explicitly account for position-dependent retrieval failures in high-stakes clinical applications.