Abstract: This survey reviews how large language models (LLMs) are transforming synthetic training data generation in both natural language and code domains. By producing artificial but task-relevant ...
LLM training data mixture optimization breaks when training pools shift — every prior proxy experiment becomes stale.