Study probes whether Qwen2.5-7B-Instruct infers Colombian identity from linguistic cues

A new arXiv preprint investigates whether the Qwen2.5-7B-Instruct model internally infers Colombian identity or related stereotypes from subtle linguistic cues in both Colombian-Spanish and English prompts. The study focuses on activation-level interpretability, using Natural Language Autoencoders (NLA) to verbalize residual-stream activations from layer 20 across four positional quartiles per prompt.

The authors constructed a dataset of 30 prompts, structured as 15 matched Spanish-English pairs. These pairs include explicit Colombian cues, implicit cues, and neutral controls to isolate potential nationality or stereotype representations before they appear in model outputs. The work emphasizes descriptive rates and qualitative evidence over statistically powered effects, framing the findings as a pilot study rather than a definitive measurement of bias.

The research situates itself at the intersection of activation-level interpretability and bias evaluation, specifically targeting underrepresented Spanish varieties. By probing latent representations in a widely used 7-billion-parameter model, the authors aim to uncover whether demographic inferences occur internally even when not explicitly stated in the input or output.