Why Decoding Sharpens Without Resolving: Lock-in and Confusion in Autoregressive Reasoning
Autoregressive reasoning traces often sharpen as they unfold — entropy falls and token probabilities concentrate — suggesting decoding is resolving uncertainty. Yet the same pattern persists after clear reasoning errors. We analyze decoding through the conditional posterior it induces over branches — equivalence classes of continuations under any chosen notion of “resolution” — and show that a mismatch between token selection and branch resolution produces two regimes: lock-in, where top tokens remain branch-evidential and reinforce commitment, and confusion, where branch-specific residuals self-average and top tokens become weakly diagnostic. Two observable consequences follow: entropy may fall while unresolved support stays elevated, and greedy token selection has opposite informational effects across regimes. We confirm these theoretical predictions across multiple open-weight models and reasoning benchmarks.
The paper, “Why Decoding Sharpens Without Resolving: Lock-in and Confusion in Autoregressive Reasoning”, co-authored by Nan Zhang and Heng Xu, was accepted at NeurIPS 2026.
View the NeurIPS 2026 poster page for this paper.
Never miss a story from us, subscribe to our newsletter