OpenDecider tests open decision models against Jev and Laya

An open decision model takes a piece of text or structured state, asks a bounded question, and returns probabilities over the allowed answers. In OpenDecider’s 27 September release, Manjunath Janardhan presents two sizes of this kind of model: OpenDecider-nano, roughly 400 million parameters, and OpenDecider-small, a 4-billion-parameter model. They are designed for tasks such as routing a support ticket or rating urgency, where an application needs a defined answer rather than a paragraph of generated text.

The release’s headline is a comparison with TypeSafe’s Jev and Laya. The useful reading is narrower than “one model beats the others”: which OpenDecider version leads depends on the dataset and whether that version trained on the dataset’s training split. The scores below are reported by OpenDecider’s author. We inspected the public model cards and comparison notes, but did not rerun the models or independently audit the datasets.

Two models for different decisions

Nano uses an Ettin encoder and gives each answer option its own marker. A small head converts the markers to scores and normalizes them into probabilities. The nano card lists a 2,048-token total input limit, about 2 GiB of running memory, and a 16 ms single-question measurement on an NVIDIA L40S. Those are model-specific conditions, not an end-to-end service latency promise.

Small is a LoRA adapter on Qwen3-4B-Instruct-2507. It reads probabilities from the next-token scores of lettered options, without generating an answer sentence. Its card lists roughly 8.9 GiB running memory and 38 ms for one question on an L40S. The adapter also requires the underlying Qwen model. An additional small-td variant is tuned for the typed-decisions training split; it should not be confused with the general small model.

The author’s comparison says nano, small-td and Laya’s typed-decisions checkpoint used the typed-decisions training split; the test split was held out. General small did not receive that tuning, which helps explain why its typed-decisions score is lower. Its score is 0.672 in the launch post and 0.671 in the repository’s current table; the table above follows the dated launch. The reported nano–Laya difference is three percentage points, with a paired bootstrap 95% interval of 1.4 to 4.4 points. That interval describes uncertainty in this test comparison, not a guarantee on a new company’s tickets.

Where the comparisons change direction

On the 200 general decisions, small’s 0.735 and Jev’s 0.730 differ by half a percentage point. The author estimates uncertainty of roughly three points for that small set, so these results are better read as similar performance there. In Laya’s application battery, Jev leads at 0.774. On phishing specifically, the author’s table gives Jev 0.897 and small 0.630. A routing or security deployment should test its own class distribution, false-positive costs and confidence thresholds rather than choose from the typed-decisions headline.

The reported general-decision expected calibration error is 0.087 for small, 0.164 for Jev and 0.162 for the Laya typed-decisions checkpoint; lower is better on that evaluation. Calibration means that predictions made at a given confidence should be correct about that often across many examples. It is useful for sending uncertain cases to a human, but a calibration score from one English dataset does not set a safe automatic-action threshold for a different workflow.

The release also includes CLM-8B in its table. Its low typed-decisions score is a poor standalone verdict on CLM: as the author notes, CLM is a contrastive reranker, with a different intended task and scoring interface from a bounded classifier. It supplies context, not a like-for-like product recommendation. The author’s latency table likewise mixes local GPU runs, Apple Silicon runs and API calls that include network time; those numbers cannot be read as a controlled hardware speed ranking.

Access and remaining limits

The nano and small cards list Apache-2.0 for the released models, and the repository provides code and saved prediction results. A live demo exposes nano for a first look. The author reports training on roughly 190,000 questions and checks for overlap with evaluation text; we have not independently verified either claim. The reported evaluations use single training seeds, and OpenDecider was evaluated in English only. Laya, by contrast, publishes a multilingual checkpoint; that does not establish a Turkish head-to-head result here.

For adoption, prepare labeled examples from the actual workflow, compare the same answer schema and scoring rule across candidates, and inspect both errors and calibration. Keep a human review path for consequential decisions. The published release gives a reproducible starting point, while its results remain the author’s measurements.

Sources checked 28 September 2026. The illustration is original editorial artwork; the official comparison image is linked for reference because its separate image reuse terms were not verified.