Paper argues pairwise preference methods overlook internal pluralism in human decision-making

A new paper on arXiv introduces a formal model of 'internal pluralism' to argue that local pairwise comparisons—commonly used to infer how people want automated decision rules to behave—may systematically misrepresent human preferences. The authors, Bailey Flanigan and Michelle Si, contend that these comparisons rely on two assumptions that often do not hold: that local comparisons are sufficient evidence for global preferences, and that people can always answer such comparisons decisively.

The paper identifies two distinct failures of forced pairwise comparison data under internal pluralism. First, priorities such as proportionality, egalitarianism, and equal treatment are inherently global: what they require in one case can depend on what happens elsewhere, so local comparisons may fail to capture them. Second, even when priorities are representable locally, tension between strongly-held priorities can generate internal conflict, producing potentially costly behavioral distortions when comparisons are forced.

To address these limitations, the authors propose allowing people to report indecision as an alternative to forced choices. Their model suggests that this approach can considerably reduce the number of queries needed to learn preferences accurately. The paper concludes by advocating for preference-learning methods that elicit underlying priorities directly, aiming for more faithful and interpretable accounts of what people value in automated decision rules.