
Evals First: Getting the Most Out of Fireworks with Vals
Vals partners with Fireworks customers to build and maintain custom evals, so they can pick the right model, train their own, and cut costs without cutting quality.


We gave nine decision models a RollerCoaster Tycoon 2 park and three in-game years to reach 1,000 guests. The highest-scoring model averaged 71%.


On our new Autoform-Hard benchmark, GPT-6.2 Astra produced a Lean proof that no graded domain has Gelfand–Kirillov dimension strictly between 2 and 3, a 2006 theorem of Agata Smoktunowicz that, as far as we know, had never been formalized. It took 94 minutes, and the argument takes a different route from the published one.

36 posts