Research proposes synthetic embeddings for LLM fine-tuning without human-readable text

Researchers propose Desired-Update-Aligned Synthetic Data (DASA), a method that fine-tunes large language models using continuous synthetic embeddings optimized via activation-gradient feedback from a frozen reference model. Unlike traditional approaches that depend on human-readable text, DASA targets useful adaptation updates rather than source-text reconstruction or linguistic fluency.

The authors evaluate DASA on six models from the Llama and Qwen families, spanning parameter counts from 1 billion to 32 billion. Across six benchmarks covering knowledge, mathematical reasoning, code generation, and commonsense reasoning, the method achieves performance comparable to natural-language fine-tuning data under matched LoRA adaptation settings. In multiple configurations, DASA surpasses natural-language data performance.

The study further compares DASA to GRADMM, a baseline method for synthetic data optimization. Under the evaluated synthesis settings, DASA provides a 3.6–4.9× speedup over GRADMM while maintaining comparable peak GPU memory usage.

The authors also examine DASA’s performance on both general-domain and task-specialized source data, indicating the method’s flexibility across different data regimes.