Order Effects in Eliciting Preferences
Abstract
Having an accurate account of preferences help governments design better policies for their citizens, organizations develop more efficient incentive schemes for their employees and adjust their product to better suit their clients’ needs. The plethora of elicitation methods most commonly used can be broadly distinguished between methods that rely on people self-assessing and directly stating their preferences (qualitative) and methods that are indirectly inferring such preferences through choices in some task (quantitative). Alarmingly, the two approaches produce systematically different conclusions about preferences and, therefore, survey designers often include both quantitative and qualitative items. An important methodological question that is hitherto unaddressed is whether the order in which quantitative and qualitative items are encountered affects elicited preferences. We conduct three, pre-registered, studies with a total of 3,000 participants, where we elicit preferences about risk, time-discounting and altruism in variations of two conditions: ‘Quantitative First’ and ‘Qualitative First’. We find significant and systematic order effects. Eliciting preferences through qualitative items first boosts inferred patience and altruism while using quantitative items first increases the cross-method correlation
for risk and time preferences.
JEL Classification: C83, C91, D01, D91
Keywords: preferences, qualitative vs. quantitative measures, risk, altruism, patience
Presented at:
- ARC seminar, Max Planck Institute for Human Development, Berlin, 2023
- Subjective Probability, Utility and Decision Making (SPUDM), Vienna, 2023
Cite this article as:
Kopsacheilis, O., & Goerg, S. J. (2024). Order Effects in Eliciting Preferences. http://dx.doi.org/10.2139/ssrn.4526269