CLM-8B: an open-weight Jev alternative for agent decisions

An AI agent needs to send an incoming message to billing or technical support. At that moment, it needs the right queue rather than a long explanation. CLM-8B offers an open-weight model for decisions like this, comparing the current situation with a supplied set of possible actions.

The project, shared on Hacker News on September 24, follows naturally from our look at Jev and Laya. That date refers to the HN submission, not a claim that every model artifact first appeared today. The useful question is where this decision-focused approach could help an agent application.

What does CLM-8B do?

CLM stands for Contrastive Language Model. It turns a situation and its candidate actions into separate numerical representations, then ranks the candidates by how well those representations match. Its training objective pulls correct state–action pairs together and pushes incorrect pairs apart.

The released implementation puts two small projection heads on top of Qwen3-8B: one represents the state, the other a candidate action. The base language model stays frozen while these heads are trained. Encoding fresh text still uses the base model. When the same candidate is reused, its representation can come from a cache.

The system therefore is not just a tiny classifier file. Alongside the roughly 75 MB head checkpoint, deployment needs the Qwen3-8B encoder. The small download should not be mistaken for the complete model’s memory footprint or hardware requirement.

A Jev-like interface with a different mechanism

The official repository exposes three question types through a TypeSafe-compatible API:

Question type	Result	Example use
Choice	A distribution over candidates and the selected answer	Route a message to editorial, subscription or technical support
Noul	A value derived from true/false candidates	Assess whether a message is urgent
Score	An expected level on a defined, ordered scale	Rate a request as low, medium or high priority

Compatibility makes it easier to experiment with similar request structures. Sharing an interface does not establish equal accuracy, context capacity or confidence calibration.

The official playground screenshot used as the cover demonstrates these three assessments on one customer message. Its percentages are project example outputs, not our independent measurements.

What sits behind the “up to 9× faster” claim?

The project’s original chart shows its largest gap in the T-Rex game: 16.5 ms for CLM-8B and 149.8 ms for Jev. The tool-calling comparison is 76.8 ms versus 125.5 ms. A single speed multiplier therefore cannot describe every task.

The success bars matter just as much. On tool calling, CLM records 95.2% and Jev 99.2%; on WikiRacing, the results are 26/30 and 30/30. In those two tasks, lower latency does not come with equal success.

The T-Rex method document adds an important qualification. A physics planner tells the model which moves are safe, and an enabled safety shield can replace an unsafe answer. Survival across five courses measures that combined system, not the model’s unaided playing ability. The CLM run used a local encoder on an RTX 4090; the Jev run used a hosted API. These client-side timings are not a pure inference comparison on identical hardware.

Coding scores: writing a solution and selecting one are different tasks

The second chart reports 81.6% on DeepSWE and 87.6% on Terminal-Bench 2.1. CLM does not solve the coding task from beginning to end in these experiments. It acts as a verifier, selecting among several solutions produced by other models.

The repository describes held-out sets of 38 DeepSWE tasks and 30 Terminal-Bench tasks. Opus 5 generated the DeepSWE candidates, and Fable 5 generated the Terminal-Bench candidates. The high scores belong to task-adapted heads; they are not presented as out-of-the-box results for the base head. Latency was measured on an H100.

For canberk.me, the interesting idea is to have a strong model produce several possible solutions and let a narrower model select among them. Producing those extra candidates still costs something. Verifier latency must be separated from the cost of the full workflow, including candidate generation.

Does 99% confidence mean 99% accuracy?

The model card explicitly says its probabilities are relative to the supplied candidate set. If the correct answer is missing, the system can still strongly favor one of the remaining options.

For example, assessing “Cancel my subscription” with only “billing” and “technical problem” available might produce a high score without establishing that the classification fits the intended workflow. The candidates and their descriptions are part of the decision problem.

Our recommendation is to design a review path and test decision thresholds against labelled examples. Turkish, mixed-language and out-of-scope messages deserve separate evaluation. We did not find a comprehensive assessment in the reviewed sources establishing readiness for Turkish production use.

Who should try it, and what should the first experiment be?

The documented setup uses Linux, an NVIDIA GPU, Python 3.10+ and vLLM. The Qwen3-8B encoder and CLM API run separately, with a browser playground also available. The code and released weights use Apache 2.0. Self-hosting provides control but does not remove hardware, operations or evaluation costs.

Our first experiment would route TR/EN reader messages, stripped of personal details, into three editorial queues and compare the decisions with human labels. It would not send email or delete anything automatically. Simple rules and a general-purpose language model would receive the same examples. We would measure confidently wrong decisions, review frequency, total response time and actual cost.

CLM-8B’s value is opening the decision layer to inspection and evaluation on our own data. We have not independently benchmarked the project. The available evidence does not establish a replacement for Jev in every workflow; it supports an open alternative worth testing on specific agent decisions.