Weekly Hallucinations: Jev, Everywhere All at Once

Claude Code now has Projects: one agent becomes the coordinator, each task gets its own cloud branch, and work continues even after you close your laptop. You’re supposed to pay for this orchestration extravaganza out of a quota that was cut by 17% on September 14 compared with what you had over the summer. By the way, have you heard of Jev?

TypeSafe has released Jev, a model that doesn’t write text. It takes a question together with a list of predefined options and returns a probability for each one. The claim is that it’s 20–200 times faster and 40–400 times cheaper than a conventional model, while output tokens are not billed at all because there simply aren’t any. Jev cannot produce free-form text and is not a general-purpose language model.

It sounds like a step back to 2018, and that’s exactly how the community reacted in the first few days: people who got into ML after ChatGPT treated it like a revelation, while those who remember the classifier era wondered what all the fuss was about. There is practical value, though. In production, a huge share of LLM calls are not about creativity but about decision points: which agent to route a request to, whether a response is good enough, which of five quotes to insert, whether to escalate a ticket or close it. At such a decision point, autoregression—that is, generating the answer word by word—turns out to be pure overhead.

I’ve already broken down the APIs of Jev-style solutions and fifteen projects built on them, so go read it! Jev wins precisely when you have correctly defined the space of valid answers in advance. If your option list is flawed, the fast and cheap model will simply be wrong quickly and cheaply.

TypeSafe did not release the weights, and within two days the community had produced six clones. Bespoke Labs built Nimble: a LoRA fine-tune of Qwen3.5-9B (adjusting a small add-on to the weights instead of retraining the whole model) on 2,676 contrastive-labeling examples. For training, they take two nearly identical texts, change one fact, and the correct answer flips. On its own held-out set of 324 examples, Nimble matched the reference answer in 90.12% of cases, versus 93.21% for Jev and 66.36% for the untuned Qwen; median response time on an H100 was 106 ms versus 246.7 ms for cloud-hosted Jev.

Laya has just 421 million parameters on a ModernBERT-large encoder. It was trained on examples with strictly defined evaluation rules so that the probability it outputs means exactly what is stated, rather than merely ranking the options. The authors report 32.8 ms per question and an accuracy of 0.766 versus 0.727 for Jev on their own typed-decision dataset. The repository picked up 6.3 thousand stars in three days. The most compact option, Kev-0.5B, is built on Qwen2.5-0.5B and runs on a MacBook Pro.

All three sets of numbers were calculated on the authors’ own data, and that is the main problem with this new category. There is no common benchmark for decision models yet, so the demos are competing on speed rather than quality, and there is no fair way to compare them with one another. Tech blogger Theo criticized one fashionable use case for Jev: running the model line by line for compacting—that is, compressing an agent’s accumulated history to fit the context window—means not understanding how that history works. If you throw away hidden pieces of reasoning, you degrade the flagship model, while any change to the history invalidates the prefix cache, making it more expensive to edit the history than to leave it alone. LangChain, meanwhile, considers browser work the best Jev use case they’ve seen: it involves lots of small, repetitive decision points.

Two new papers on model harnesses came to different conclusions. In Is Bash All You Need?, Microsoft compared five tooling setups on TheAgentCompany and APEX-Agents, two benchmarks of office and enterprise tasks for agents, using Opus 4.8 and GPT-5.5. Plain bash beat typed tool catalogs by 21.8–24.5 points on the first benchmark and 4.8–7.4 on the second, while using 19–72% fewer tokens. Adding typed tools on top of bash did not improve the results at all.

The second paper, An Empirical Study of Harness Design for Coding Agents, ran 176 harness configurations across four models on SWE-bench Verified and reaches a more nuanced conclusion. Models that are good at bash really do work more cheaply with no predefined tools at all, while weaker models benefit from a typed toolset. Planning behaves similarly: for weaker models it props up accuracy, while for stronger ones it saves money. I went through the Codex, OpenCode, and Pi source code over the summer and wrote back then that harness architecture matters more than the model itself.

The format war over agent instruction files is finally over. Claude Code 2.1.277 added support for reading AGENTS.md: if a project does not contain CLAUDE.md, the agent takes its instructions from there. The mode can be switched through /config, although this support does not yet work on Bedrock, Vertex, or Foundry. Simon Willison, who has long chronicled the tooling ecosystem around LLMs, immediately pointed out the practical result: there is no longer any need for stub files that did nothing but point to one another. AGENTS.md is supported by Codex, Cursor, Copilot, Jules, and a couple dozen other tools, so there was no reason left to keep holding out for a proprietary format.

Claude Code also got Projects. A single conversation becomes a coordinator: you drop in a bug report, trace, or task list, and Claude creates a separate branch for each one. Every branch is a full cloud Claude Code session, the branches run in parallel, keep working after you close your laptop, and you can monitor and steer them from your phone. For now this is a public beta on Pro and Max, rolling out gradually starting with people who had already used cloud sessions; Team and Enterprise do not have it yet.

You’re supposed to pay for this orchestration extravaganza out of a smaller quota. The promotion with 50% higher weekly limits, which had been running since May 13, ended on September 13, and from the 14th the weekly Claude Code limit was set at 25% above the pre-promotion level for Pro, Max, Team, and per-seat Enterprise plans. Compared with what people had been using all summer, that is a 17% cut, and r/ClaudeCode reacted accordingly. In a neighboring thread, the owner of a Max 20x plan shows a screenshot where the weekly all-model limit is at 100% while Fable is at 78%, and complains that a hundred dollars’ worth of credits disappeared in half an hour. The advice in the comments is unanimous: don’t buy extra credits; they burn faster than you think.

Personally, I canceled my $200 Claude Max subscription. Before that I kept it alongside ChatGPT Pro for the same $200 and split different tasks between the two. At first it seemed like I was choosing between Astra/Sol and Fable/Opus. Then I realized that, all else being equal, the choice comes down to something else: write code for three days (Claude) or four, with the possibility of the limits resetting on top of that (ChatGPT).

The RTX 5090, with an MSRP of $1999, has vanished from U.S. retail, while third-party sellers are asking as much as $9500. On r/LocalLLaMA, people write that the nearest Micro Center has only RTX 6000 Pro cards left among the large GPUs, at around $14000, while a laptop with a mobile 5090 is selling a year later for almost what it cost new. The reasons are boring and persistent: a GDDR7 memory shortage has collided with inference demand, and every card sent off to run models is one less card on a store shelf. NVIDIA has published the specifications for the RTX PRO 5500 Blackwell: 84 GB of ECC GDDR7 memory, 1398 GB/s, PCIe Gen 5 x16, up to 600 W, and MIG for two fully isolated instances, meaning the card can be split into two independent virtual GPUs. The status is "coming soon," there is no price, and the community is wondering whether it will turn out to be a binned RTX PRO 6000 die.

Dario Amodei, CEO of Anthropic, wrote a personal post about slowing down built around three points: embedded third-party evaluators, coordination among democracies, and an attempt to reach agreements with authoritarian states. Anthropic is adopting the first point unilaterally right now: outside evaluators are being promised desks in the company’s offices, badges, corporate laptops, and access to work environments, tools, and permissions roughly on par with internal risk-evaluation teams. He is more cautious about China: first maintain the technological lead by restricting chip sales, protecting model weights from theft, and combating unauthorized distillation, and only then move toward global agreements.

The AI Evaluator Forum, an association of evaluators that includes METR, RAND, Transluce, SecureBio, and others, has already published minimum conditions for embedding evaluators, signed by more than a hundred people, including Nobel laureate Geoffrey Hinton. There are five points: full editorial control and no payments tied to conclusions; multiple organizations with different areas of expertise rather than just one; narrow NDAs and the right to publish findings; protection from legal pressure and stable funding even after unfavorable reports; and privileged-employee-level rights with direct access to people and systems. It is also explicitly stated that embedded evaluators cannot solve every oversight problem and do not replace external review. A week ago I covered four Anthropic incidents in which Claude reached real third-party systems during testing and registered a malicious package on PyPI, after which the investigation was handed to METR. Now METR is signing a document laying out the conditions under which looking from the inside makes sense at all.

Stay curious.

I write about artificial intelligence, language models, and tools for developers. I test models and services on real-world tasks and share my findings in my Telegram channel.