Study identifies sparse neuron subsets in BERT that drive AI-text detection

A new arXiv preprint studies which neurons in a frozen BERT-base-uncased encoder support AI-text detection across six generators using the RAID benchmark. The work applies the L1-to-L2 sparse-probing protocol to all 9,216 CLS hidden-state dimensions (12 layers × 768 dimensions), treating each dimension as a neuron.

The authors report recovering a stable set of under 1% of neurons per generator, consistent across folds and random seeds. A linear probe restricted to this sparse set retains most of the full-feature detection accuracy, indicating that the signal is concentrated in a small subset of neurons.

Bidirectional activation patching further confirms the causal relevance of these neurons: patching in either direction flips predictions an order of magnitude more often than size-matched random sets, demonstrating their functional importance for detection.

Mean-ablation experiments show that removing these neurons leaves accuracy largely intact, suggesting the detection signal is redundantly distributed across the network rather than localized to a single critical set.

Cross-generator analysis reveals a bipartite structure: instruction-tuned generators concentrate 30–36% of stable neurons in BERT’s final layer, while both base generators fall below 14%, consistent with a layer-12 footprint of post-training alignment.

Leave-one-family-out evaluation indicates that the selected sparse neuron sets retain 86–94% of the full-feature ceiling on unseen generator families, implying that a detector could operate on a small fixed subspace without re-identifying neurons for each new generator family.