AI Advisor or Human Expert? Regret, Disappointment, and Future Reliance After Following or Rejecting Advice

Abstract

Decision making is increasingly supported by artificial intelligence (AI) systems as well as human experts. This study examined whether advice source (AI algorithm vs. human expert) and the decision to follow or reject advice influenced reported regret, disappointment, and willingness to use the same source again after a negative outcome. Participants (N = 212) completed three hypothetical decision scenarios concerning airline tickets, a winter jacket, and a stock market investment in a 3 (scenario; within subjects) × 2 (advice source; between subjects) × 2 (advice followed vs. rejected; between subjects) mixed design. In the advice-followed condition, the recommendation subsequently proved incorrect, whereas in the advice-rejected condition, the recommendation retrospectively proved correct despite the negative outcome. Regret and disappointment differed significantly across scenarios, but neither emotion showed a main effect of advice source. Significant advice source × advice-taking interactions were found for both emotions; however, the direct comparisons between algorithmic and expert advice within the advice-followed condition were not significant and did not support the predicted differences. Participants reported greater overall willingness to reuse expert than algorithmic advice. Nevertheless, the predicted lower willingness to reuse algorithmic rather than expert advice specifically after following an incorrect recommendation was not supported. The largest effect concerned the advice-taking condition: willingness to reuse the source was substantially higher after rejecting a recommendation that subsequently proved correct than after following a recommendation that subsequently proved incorrect. Because advice-taking was confounded with retrospective advice accuracy, their independent effects could not be determined. The hypothetical scenarios and low reliability of the Regret and Disappointment Scale indices limit the strength and generalizability of the conclusions.