A model that cannot say the wrong word.
1. The problem with prose
We shipped a reranker in loomcycle that, on paper, should have made document search measurably better. The reranker read a short list of candidate chunks, decided which was most relevant to the query, and wrote the new order as prose. Something like: "The most relevant is chunk 3, then chunk 1, then chunk 5, then chunk 2, then chunk 4."
It worked. Mostly. Occasionally the reranker would emit a chunk id that was not in the input list. Occasionally it would add commentary ("Based on my analysis, I think...") that we then had to parse past. Occasionally it would forget to answer the question it was asked and reply with a summary of the candidates instead. The failure rate was low enough to look like noise and high enough to show up on a long run.
A model writing prose has infinite room to be wrong. The fix is to use a model that does not write prose.
A decision model is a small model trained to answer a typed question with calibrated probabilities over a fixed set of options. It writes no prose. It cannot cite a nonexistent option. It cannot forget to answer. v1.104 shipped memory.reranker.kind: decision using the Ollama nimble model as a drop-in replacement for the prose reranker: measured LoCoMo recall@5 lifted from 0.588 to 0.740, ConditionalQA R@5 0.944 vs 0.935 listwise at about a third of the latency. v1.107 made the primitive general: a Decision tool any agent can call, over HTTP, gRPC, MCP, TypeScript, Python. Three question types (choice, noul, score). Model aliases now carry kind: chat | decision | embedder; a wrong-slot wire fails config load with a named error.
That is the whole arc in a paragraph. The rest of this piece says what each piece does and why it is the way it is.
2. What a decision model is
A chat model is trained on next-token prediction over text. It can write anything, including the wrong thing. When you ask it "which of these five documents answers the question", it responds with text that may or may not reference the right id, in the right format, in the right order.
A decision model is trained on a different task. You give it a question and a fixed set of answer options. It returns a probability distribution over those options. The model cannot emit something that is not an option because the output space does not include "something that is not an option."
The output is not text. It is a vector of probabilities.
Three question types ship in the primitive:
choice: pick one of 2 to 26 options. Returns a probability per option.noul: yes-or-no. Returns the probability of "yes."score: ordered levels (for example low / medium / high / critical). Returns the expected position on the ordinal scale.
You can send 1 to 64 questions per call. The model answers each independently.
3. What did swapping change?
The first production use was the memory reranker. Before this release, our reranker was a prose model (listwise rerank: give it the candidates, it writes the new order). We swapped it for a decision model on the same inputs. Same candidate set, same query, same top-k. One variable.
The LoCoMo number is the big one. Reranking memory recall (not document search; the recall path over the user's facts, notes and documents) at k=5 went from 0.588 to 0.740. That is the test where the prose reranker's occasional-wrong-id and occasional-commentary failures mattered most, because downstream the answerer saw whatever the reranker chose and could not recover from a reranker mistake.
ConditionalQA at R@5 is a cleaner comparison: 0.944 for decision vs 0.935 for listwise. The accuracy difference is small; the latency difference is about three times. On a corpus where the prose model was already near ceiling, the decision model is cheaper for the same answer.
4. Reranking recall, not just documents
v1.105 added a memory.reranker.sources: [documents, facts, notes, traces] setting. Before, the reranker was a document-search thing. Now it can reorder a recall over any mix of fact rows, note rows, and the traces attached by recall_attach_traces. On the LoCoMo memory benchmark, the recall-reranking lift is what the 0.588 → 0.740 number above measures.
Also in v1.105: the listwise default candidate pool bumped from 20 to 40. The decision model does not need the smaller pool to stay fast enough, so recall gets a wider net.
5. The Decision tool
Internal uses of a decision model are nice. What v1.107 ships is the primitive itself, exposed as something any agent can call.
Inside a run, an agent that holds a Context grant gets a Decision tool. From outside: POST /v1/_decide on HTTP, Decide on gRPC, decision on MCP, decide() on the TypeScript and Python clients. Same call shape in all five.
// a classification call
await Decision.ask({
question: "What is the sentiment of this review?",
context: "The waiter was slow but the food was great.",
type: "choice",
options: ["negative", "neutral", "positive"],
})
// returns: { probabilities: { negative: 0.08, neutral: 0.19, positive: 0.73 } }
// a yes/no call
await Decision.ask({
question: "Does this chunk answer the user's question?",
context: "...retrieved chunk...",
type: "noul",
})
// returns: { probability_yes: 0.91 }
The use cases we hit first are classification, verification (does this chunk really answer the question?), and routing (which of three skills should handle this request?). All three were previously built with a prose model and a parser that frequently had to deal with the model answering a question it was not asked.
6. Model aliases now carry a kind
When model aliases could be any kind, nothing stopped an operator from accidentally wiring a chat model into a slot that expected an embedder, or a decision model into a chat slot. The config loaded; the first call failed with an uninformative error.
v1.107 added kind: chat | decision | embedder to every model alias. A slot that needs a decision model refuses to accept a chat model at config-load time, with a named error naming the slot and the alias. Same for the other combinations. A deployment that misconfigures a model slot now fails in the config file, not three turns into a run.
7. What this does not fix
Three limits worth naming.
Only Ollama 0.35+ serves decision models today. The models that ship with support are nimble, clef, clef-flash, and tev1. Other inference backends (vLLM, llama.cpp, hosted APIs) do not serve the decision-model API yet. If you are running on a backend that does not, the fallback is the prose reranker that was already there.
Probabilities are not cross-call calibrated. A 0.8 on one question is not the same confidence as a 0.8 on another. You can compare probabilities within one answer (A got 0.8, B got 0.15, pick A). You cannot compare across answers without calibration. If you want to gate on probability ("only use the model's answer when it says yes with confidence above X"), measure the actual confidence distribution on your corpus first, then set the gate.
Refused-as-too-large retries at half then quarter character budget. If the context plus options plus question exceeds the model's input budget, the runtime retries at half the character budget, then a quarter. If the three attempts all refuse, the call fails. For long contexts this is not visible; for very long ones it is. The retry chain is logged.
8. What this makes possible
A verification step in a cascade (see the retrieval-quality post) used to run a small chat model that read the retrieved material and wrote "yes this answers" or "no it does not." That chat model occasionally wrote neither. Now the verifier is a noul decision call and the output is a probability.
A classification agent (sentiment, routing, triage) can replace its prose model with a choice decision call. Same inputs, no parsing, no retry-on-bad-output, roughly a third of the latency on the shapes we tested.
A code-js orchestrator that routes work between skills can call Decision.ask as a tool and get a probability-ranked list of skill names it can safely dispatch to.
Decision models are a primitive. The arc that keeps building on them is detached sub-agent runs, where a poll-mode parent can afford to run a verifier on every child's answer because the verifier is cheap. The next two posts cover teams as one definition and the canvas you build those teams on.
A prose model can answer any question in any way. That is its gift and its hazard. For the questions whose correct shapes are already known (yes-or-no, choose one of N, assign a score), the hazard outweighs the gift. A model that cannot say the wrong word has a smaller surface to go wrong on, and a shorter path from question to answer.
The Decision tool is one answer to a plain question. We built it so the parts of an agent that pick, verify, rank or route stop fighting their own output. The question goes in with its options named. The probability comes back. The agent gets on with the next step.