Hugging Face and Liquid AI demonstrate fine-tuning recipe to improve structured-output compliance in a 350M-parameter model

Hugging Face and Liquid AI published a step-by-step guide demonstrating how to fine-tune a 350M-parameter model for better structured-output compliance using Group Relative Policy Optimization (GRPO) and the TRL library.

The authors fine-tuned LiquidAI/LFM2.5-350M on a public instruction-following dataset with structured-output targets, then evaluated the result on the open IFStruct benchmark. The full pipeline is designed to run on free-tier Colab or Kaggle GPUs, with evaluation possible on a MacBook using llama.cpp.

Before fine-tuning, the base model achieved 22.6% on IFStruct under a local llama.cpp/BF16 serving setup, close to the 21.1% reported by the IFStruct release. After 100 GRPO training steps with roughly 500 samples, performance rose to 29.7%.

The guide includes a public GitHub repository with the training notebook, dataset preprocessing, LoRA configuration, reward functions, and evaluation scripts. The authors emphasize that the goal is not to recreate the RL-trained IFStruct score, but to show that task-specific fine-tuning can deliver meaningful improvements in schema compliance for smaller models.

The authors also document common error modes observed in the base model’s outputs—such as missing required fields, wrong item counts, type mismatches, and extraneous fields—and describe how the fine-tuning targets these issues via structured-output rewards and prompt augmentations.