The Optimization Trap: How Artificial Intelligence Diminishes Decision Rationality in Organizations

Abstract

A common assumption holds that artificial intelligence improves organizational decisions because it processes more data, computes faster, and behaves more consistently than human decision-makers. This article argues the opposite for a large class of consequential decisions: AI frequently decreases decision rationality in organizations, even as it improves prediction accuracy, processing speed, and internal consistency. The problem is not a malfunction. It is that AI succeeds at something narrower than what organizations actually need, while making that narrow success resemble the whole picture. Drawing on Simon’s account of bounded rationality and the behavioral tradition that followed, the paper distinguishes a rich conception of organizational rationality – one requiring contextual sensitivity, interpretive flexibility, value awareness, and reflexive judgment – from the thin, optimization-based notion that dominates managerial discourse on AI. It then identifies four interconnected mechanisms through which algorithmic systems erode the richer capacity: proxy optimization, which substitutes measurable targets for meaningful goals; automation bias, which compresses human judgment around algorithmic defaults; decontextualization, which strips cases of the particularity that good decisions depend on; and the suppression of reflexive judgment, which prevents organizations from questioning their own premises. These mechanisms produce cumulative organizational consequences: weakened accountability, diminished contestability, and reduced capacity for adaptive learning. The argument is not that organizations should abandon AI, but that they should stop treating it as a rationality upgrade. AI is a capable tool for narrow computational tasks and a poor substitute for organizational judgment. The organizations most at risk are those that have forgotten the difference.