Researchers propose FINDERS framework to improve user representation learning with diffusion language models

Current LLM-based user encoders serialize heterogeneous behavioral logs and compress them into static representations via autoregressive prediction, which imposes unidirectional constraints that hinder cross-modal interactions and the joint preservation of multi-source signals. These static representations often fail to meet the distinct evidence requirements of diverse downstream scenarios, limiting their utility in dynamic environments.

To address these limitations, the authors propose FINDERS (FactorIzatioN of Diffusion statEs for Routing across Scenarios), a framework that repurposes the iterative denoising process of diffusion language models (dLLMs) as a multi-granularity reconstruction space. The framework consists of four core operations: denoise, factorize, route, and compress, which collectively enable more flexible and adaptive user representation learning.

FINDERS introduces learnable meta-slots to organize complementary denoising states into transferable cross-modal semantics and modality-specific behavioral factors. This design allows the framework to dynamically emphasize relevant slot directions based on scenario queries through an input-level residual pathway, prior to cross-source fusion and final compression.

To support grounded supervision, the authors construct BehaviorQA, a large-scale corpus containing explicit behavioral rationales. This dataset provides the necessary supervision signals for training the router and factorization components of FINDERS, ensuring that the learned representations are interpretable and aligned with real-world user behaviors.

The framework is evaluated through extensive experiments and online A/B tests across ten industrial scenarios, where it consistently outperforms state-of-the-art methods in both unified and scenario-adaptive settings. The results indicate that FINDERS not only improves performance but also enhances adaptability to diverse and dynamic user modeling tasks.