Pre-release reasoning in open-weight LLM shows persistent wrong answer commitment despite task constraints
Study finds Qwen3-8B overwhelmingly commits to incorrect answers before generating reasoning, even when task premises invalidate the choice; activation probes detect early bias not explained by lexical or text-recovery effects.
1 source · cross-referenced
- A minimal probe task shows Qwen3-8B overwhelmingly recommends walking when only driving satisfies the premise ('car must be at the car wash').
- Across five system-prompt conditions (210 rollouts), wrong commitments occurred in 85–100% of sampled rollouts and 100% of greedy rollouts, including in both thinking and non-thinking modes.
- Activation-level probes before answer emission read out 'walk' in 68% of cases versus 17% in neutral contexts, with early 'walk'-leaning also present in rollouts that eventually answer 'drive'.
- The authors frame results as preliminary due to small sample sizes and lack of significant positional gradients within rollouts.
Researchers report that the open-weight model Qwen3-8B frequently commits to an incorrect answer before generating reasoning that justifies it, even when the task premise invalidates the choice. In a minimal probe task—"I want to wash my car. The car wash is 100 meters away. Should I walk or drive?"—only driving satisfies the premise because the car must be at the car wash, yet the model overwhelmingly recommends walking.
Behavioral reproduction across five system-prompt conditions (210 rollouts) showed wrong commitments in 85–100% of sampled rollouts per condition and 100% of greedy rollouts, occurring in both thinking and non-thinking modes. Increasing the thinking budget to 4,096 tokens did not correct the error.
Using a pretrained, training-free activation oracle to probe hidden states before the answer text is emitted, the authors find that "walk" read-outs exceed a neutral-context baseline (68% vs. 17%). The effect is statistically significant for both walk-committing and drive-committing rollouts (p = .005, Fisher exact). Notably, rollouts that eventually answer "drive" still read as walk-leaning before commitment in five of six cases.
The oracle’s default on unrelated content is "drive" (83%), indicating that the early "walk" read-outs are not due to lexical bias. Stratifying by literal occurrences of "walk" or "drive" shows the effect is not text recovery: spans containing "drive" still read out as "walk." In balanced lexical fields, per-rollout walk-majorities beat a per-prompt neutral baseline (15/22 vs. 1/8, p = .01), and all six drive-committing rollouts read as walk-leaning (p = .002).
The authors caution that samples are small and the within-rollout positional gradient is not statistically significant (p = .34), framing the activation-level evidence as preliminary. They also note that question wording alone can shift outcomes in a positive control (from 2/16 to 11/16), and that negative oracle results are uninterpretable without per-wording positive controls.
- Jul 21, 2026 · arXiv cs.CL
Researchers propose RIMS, a preference optimization framework for small-scale LLM retrieval-augmented generation
Trust79 - Jul 21, 2026 · arXiv cs.AI
Researchers release open dataset and compact 1D CNN model for recognizing affective touch in soft robotic companions
Trust79 - Jul 21, 2026 · arXiv cs.AI
Researchers propose audit framework for rater state bias in RLHF preference data
Trust79