Feasibility study compares standalone LLM and agentic pipeline for explaining ICU mortality predictions

A revised feasibility study published on arXiv compares a standalone large language model (LLM) against a four-step agentic pipeline for explaining machine-learning–based ICU mortality predictions. The work uses the eICU Demo dataset, which includes 2,353 ICU stays with an 8.1% mortality rate.

The standalone LLM and the agentic pipeline were evaluated on a stratified subset of 38 cases. The agentic pipeline eliminated outcome leakage entirely, whereas the standalone LLM produced one explanation with explicit outcome leakage. Among the 14 cases overlapping with a SHAP review subset, the standalone LLM showed higher SHAP alignment (mean Jaccard 0.171 versus 0.077) and higher direction consistency (92.9% versus 78.6%).

The agentic pipeline, by contrast, demonstrated higher guideline grounding (0.762 versus 0.143), higher value specificity (0.236 versus 0.143), and slightly higher plausibility (0.700 versus 0.671). The authors note that while agentic decomposition may improve safety-relevant grounding and patient-specific detail, it should be paired with attribution-based checks before deployment in high-stakes risk explanation contexts.