Apple proposes environment-free synthetic data generation for training API-calling agents

Apple’s Machine Learning Research team describes an environment-free synthetic data generation approach for training API-calling large language model (LLM) agents. The method addresses a bottleneck in scaling agent training: the need for fully implemented environments with executable APIs and pre-populated backend databases to collect high-quality trajectories.

The proposed approach uses LLMs as on-the-fly digital world models. Given only API specifications, an LLM first generates diverse tasks that are solvable with the provided APIs. A teacher agent then iteratively solves each task while a separate LLM simulator generates coherent synthetic API responses conditioned on the task context and simulation history.

An LLM judge filters the resulting trajectories to ensure dataset quality. The authors evaluate the method on the AppWorld and OfficeBench benchmarks, which include both information-retrieval and state-changing tasks.

Fine-tuning models on the synthetic data yields significant performance gains, demonstrating that effective supervision for API-calling agents can be generated without any executable environment. The results suggest LLM-based API simulation is a practical, scalable solution for training agents across diverse API ecosystems.