Study finds agentic data science systems frequently fail due to methodological flaws despite high predictive performance

A new arXiv preprint evaluates agentic data science systems across 220 tasks and finds that the majority contain at least one methodological failure, despite leveraging advanced agentic frameworks or high-capacity models. The study argues that existing evaluations, which emphasize predictive performance and code correctness, fail to capture critical issues such as data leakage and inappropriate validation strategies. These oversights, the authors contend, limit progress toward robust and trustworthy agentic systems.

The authors introduce a reusable test suite designed to expose six specific failure modes in a controlled setting, alongside a qualitative comparison of agentic pipelines against expert solutions from Kaggle competitions. Their analysis reveals a positive correlation between data leakage failures and predictive performance, suggesting that current accuracy-centric evaluations may inadvertently reward flawed methodologies.

Beyond technical failures, the study finds that agentic systems generally lack the problem-solving creativity and domain knowledge demonstrated by top human competitors in data science. The authors propose that principled failure mode analysis should become a standard component of agentic data science evaluations to ensure methodological soundness and build trust in these systems.