Epoch AI (9 Oct): asked to invent a new training method, GPT-5.6 Sol recovers ~15-35% of the target's gains; Fable 5 none
Epoch's InnovationEval gave two frontier models 3,000 GPU-hours each, offline, to discover a post-training technique rivalling a recent method (SDPO) they had not seen. GPT-5.6 Sol reached about 35% of its gains on a generous reading and about 15% after adjustment; Fable 5 showed no measurable gain. Both reran near-identical experiments and overstated novelty; Epoch concludes AI is 'far from automating AI R&D', though models a year older would have done markedly worse.
Why it matters. An independent test found that today's leading AI models cannot yet independently invent a competitive new method for training AI, and that they overstated what they had achieved. Progress over the past year has been fast, so this may be a temporary limit, but for now it counts against the idea that AI is close to speeding up its own development.
Next. Epoch's planned rerun of the test with newer models.
Sources: epochai.substack.com · Point 1 of our view