Skip to content
NewsModel release

Liquid AI's d1-ultra claims an 8-point benchmark lead as the fourth decision model to ship in two weeks

· by Pondero Newsdesk

The short version

Liquid AI released d1-ultra on October 15 with a claimed 8-point lead on the Artificial Analysis Decision Index, days after Microsoft, OpenAI, and AWS shipped rival decision models, two of them at the identical $0.042-per-million-token price.

Liquid AI's d1-ultra claims an 8-point benchmark lead as the fourth decision model to ship in two weeks

Liquid AI's October 15 release of d1-ultra made it the fourth vendor in two weeks to ship a model that skips generated text entirely and returns a probability for a fixed set of answers instead, per Liquid AI's own announcement. Liquid says d1-ultra beats every other decision model it tested on the Artificial Analysis Decision Index by at least 8 points. A week before that launch, Liquid's own Head of Post-Training told a podcast audience that the category's benchmarks "can't really be trusted yet," according to ThursdAI.

What

d1-ultra takes text, images, or both, plus a developer-defined question, and returns a calibrated probability in a single forward pass with no output tokens generated, per Liquid AI's blog post. It answers three question types: a yes-or-no probability, a single label picked from a list of choices, and a position on a numeric scale. Liquid built d1-ultra on a larger open-weight base than its earlier d1 model and compared the two against TypeSafe's Jev 1.14 across 112 evaluations, reporting that d1-ultra wins on both accuracy and calibration, meaning its confidence scores track its actual hit rate more closely than rival models' do, per the same post. The model is live now through the Liquid API at console.liquid.ai, billed at $0.042 per million input tokens with no charge for output tokens; Liquid lists Vercel and OpenRouter access as "coming soon." Pricing follows the same input-token-only logic AWS and OpenAI already use for their own decision models: Liquid charges 1.5 tokens per 32x32-pixel image patch, so a 1024x1024 photo costs 1,536 tokens before the question text is even counted, per the blog post.

Four vendors are now selling nearly the same model at nearly the same price

Liquid's launch lands in a category that filled up fast. AWS's Strands Labs shipped the open-weight Strands Decider 2B on October 1, a free download that runs locally in roughly 115 milliseconds (Pondero's coverage of the Strands Decider launch has the detail). OpenAI followed with a hosted Decisions API in beta on October 6, priced at $0.10 per million input tokens (Pondero's coverage of the OpenAI Decisions API has the pricing and latency numbers). Microsoft then launched Decision-1 on October 13, pricing it at the exact same $0.042 per million input tokens Liquid would charge two days later, after testing it across 36 benchmarks covering nearly 150,000 held-out questions and reporting it ran about 2.5 times faster than the next-best model it compared against, per Microsoft's blog post as reported by InfoWorld. For a team picking a decision model today, the architecture question is already settled: fixed answer set, single forward pass, no output tokens. What is left to differentiate on is whose probability score can actually be trusted, and that is the one claim none of the four vendors has let an outside lab verify yet.

Context and reactions

AWS credited TypeSafe's Jev with starting the category earlier this year, after Jev briefly topped the field's shared benchmark, JevBench. Liquid instead measures itself against the newer Artificial Analysis Decision Index, a different public leaderboard, which means the four vendors now shipping competing decision models are not even converging on the same scoreboard. Forrester analyst Charlie Dai told InfoWorld that specialized decision models like Microsoft's "can significantly reduce cost and latency" for routing and classification work that currently burns a full LLM call, framing the category's appeal in plain operational terms rather than benchmark bragging rights. Liquid's own skepticism, voiced by Head of Post-Training Maxime Labonne on ThursdAI eight days before d1-ultra shipped, was blunter: asked about the wave of rival launches, he said "it's really a wild world, and you cannot really trust these benchmarks," per the ThursdAI interview. That comment predates d1-ultra and was made about the category broadly, not Liquid's own new model, but it is a notable caveat from inside the company now claiming the category's top score.

What to watch next

Watch for an independent lab to run d1-ultra, Decision-1, and Jev 1.14 against a shared benchmark rather than each vendor's own preferred leaderboard, which would be the first real test of Liquid's calibration claim. Also worth tracking: whether Liquid's promised Vercel and OpenRouter listings land on the same $0.042 pricing once they go live, and whether a fifth vendor enters before either benchmark settles into the field's default.

Sources