Decision models after Jev: who shipped what, and how to read the claims

Decision models went from one product to a crowded field in under three weeks. TypeSafe introduced Jev on September 15. Since then Strands, Cloudflare and Perplexity have each released an open model, and OpenAI has announced a Decisions API. This is a guide to the field as of October 2: who shipped what, what each says about itself, and how to read numbers that do not line up. We read the model cards and announcements; we did not run any of the models.

A decision model takes a situation and a set of allowed answers and returns a probability for each, instead of writing text. We explained the idea in Jev and Laya-MLX.

The field in one table

All figures are what each maker reports, not independent results.

Model	Maker	Base and size	Access	What it reports
Jev	TypeSafe	not disclosed	closed, hosted API	The reference point for the rest
Strands Decider 2B	Strands team	Qwen3.5-2B-Base	Apache-2.0	72.3% on JevBench’s public set, about 115 ms median
Clef and Clef-Flash	Cloudflare	Qwen3.8-27B, Qwen3.5-9B	Apache-2.0, Workers AI	61.2 on the Decision Index (self-reported), 38.8 ms for Flash
pplx-decider-v1-27b	Perplexity	Qwen3.8-27B	Apache-2.0	85.71% against Jev’s 84.51% over 11 benchmarks
JEV-27B	autotrust	Qwen3.8-27B	Apache-2.0	Six-benchmark mean 84.07 against 83.85 for Jev 1.13
OpenAI Decisions API	OpenAI	built around Luna	limited preview	No figures in the announcement

Earlier entries in the same cluster: OpenDecider, CLM-8B, Jeeves with reasoning, and Jeff at 0.8B. A Hugging Face search for “decider” and “jev” sorted by newest also returns dozens of repositories created since September 29: GGUF, EXL3, ONNX and NPU conversions, plus domain-specific variants, most with no likes yet. Many look like conversions of existing models.

Three announcements we had not covered

Perplexity’s pplx-decider-v1-27b

Perplexity published the model late on October 1 (UTC). It is fine-tuned from Qwen3.8-27B under Apache-2.0, with the language-generation layer replaced by a decision readout of up to 255 options. The card lists about 49 GiB of weights, so it needs a large GPU, and it accepts images.

Its table compares the model with Jev and with the unmodified Qwen3.8-27B on 11 benchmarks. The overall figure favours Perplexity: 85.71% against 84.51% for Jev and 74.76% for the base model. The rows tell a more mixed story. Marked in the card’s own bolding, Jev wins six rows (WinoGrande, JudgeBench, BBH, JevBench’s public hard tier, Belebele, TruthfulQA), Perplexity wins four (FinancialPhraseBank, RAGTruth, TabFact, Circa), and ContractNLI is a tie with the base model. The biggest gap in Perplexity’s favour is RAGTruth (88.80% against 77.27%); the biggest against is BBH (82.80% against 94.27%). Two caveats from the card: the results “were measured through the Perplexity API,” not from the downloadable weights, and the card does not say how the Jev column was obtained.

One more detail. The release manifest describes the checkpoint as an “AutoJev 27B curated SFT checkpoint” and ships autojev training and serving code. autotrust’s card mentions an unrelated AutoJev-27B by another author. We could not confirm how these are related, and Perplexity’s release does not explain it.

autotrust’s JEV-27B

autotrust took a different route: a distillation of Jev 1.13. Its System 1 head imitates Jev’s probabilities (mean KL 0.017 on held-out questions Jev labelled), and the same weights keep an untouched Qwen3.8-27B for ordinary text generation. In its own runs on six public benchmarks, JEV-27B’s mean is 84.07 and the hosted Jev 1.13’s is 83.85, a 0.22-point gap. JEV-27B wins four of the six, Jev two, and baseline scores for the other open models come from another group’s evaluation, not autotrust’s own.

The card is unusually frank about limits. It says the model “mirrors” Jev 1.13, “including its mistakes”, that it is English-centric, that the speed comparison with the hosted API is not like for like (no network hop on its side), and that it is not for high-stakes decisions. Since the labels come from a closed model’s outputs, anyone building a product on it should also read how TypeSafe’s terms treat that use; we did not check.

OpenAI’s Decisions API

OpenAI’s DevDay recap describes it as focusing Luna’s intelligence on user-defined questions with finite, pre-defined answers, with text or image context, for classifying content, routing requests or choosing an agent’s next action. It is in limited preview, with a broad release planned “in the coming days.” The recap gives no price, latency or request format. TechCrunch says it had not seen developers testing it yet, so nothing about its accuracy or calibration can be assessed.

How to read the numbers

• Three Jev scores, three benchmarks. Perplexity’s table puts Jev at 84.51, autotrust’s run at 83.85, and Cloudflare’s Decision Index chart at 57.9. They are different suites with different samples. A model that tops one list says nothing about another.
• Who ran it matters. Cloudflare’s own chart separates “self-reported” from “board-validated” results. Perplexity measured through its own API; autotrust’s footnote says its figures are self-reported. None of the 27B claims has an independent run yet.
• A shared base. Clef, pplx-decider, JEV-27B and the AutoJev checkpoint all start from Qwen3.8-27B. Differences between them come from training data and recipe, not size, which is also why small gaps (0.22 points, 1.2 points) deserve caution.
• A mean hides rows. Perplexity’s overall lead comes from four wins and a tie against Jev’s six. Read the task closest to yours, not the average.
• The unmeasured parts. Calibration on your traffic, Turkish and other languages (autotrust’s corpus is English, OpenDecider was tested in English only), and latency on your hardware are not covered by any of these tables.

Choosing one

• No GPU, or you want the simplest path: a hosted API (Jev now, OpenAI’s once it is generally available).
• Small and local: Strands Decider 2B or Jeff, which run on modest hardware. Expect weaker results on long, reasoning-heavy inputs; Strands’ own card says so.
• Largest open models: Clef, pplx-decider and JEV-27B need a large GPU. Test the two or three closest to your task on a labelled sample of your own data.
• When reasoning matters: Jeeves trades latency for thinking.

Whichever you pick, set your confidence threshold on your own labelled examples, and send the uncertain cases to a stronger model or a person. The Strands and autotrust cards both say this.

What to watch

• Independent evaluations, such as Decision Index and JevBench submissions, for Perplexity and autotrust.
• OpenAI’s general release, with a price, latency and an actual request format.
• TypeSafe’s response. TechCrunch quotes its CEO saying that making these models actually smart is harder than people assume.
• Calibration under drift. The probabilities are the product; the interesting results will be the ones that test whether they hold on new data.

Further reading

• llama.cpp now serves decision models through /v1/systemone
• Red Hat: Jev is competitive with guardrail baselines, not clearly better

Editorial review: October 2, 2026 (AI). Prepared from the Hugging Face model cards and release files for pplx-decider-v1-27b and JEV-27B, OpenAI’s DevDay recap, the Cloudflare and Strands posts covered in our earlier articles, and TechCrunch’s reporting, read on the same day. All benchmark figures are the makers’ own and were not independently verified; we did not run any of the models. The scorecard image is autotrust’s, reproduced with credit and a link to the source.