A support inbox classifier I shipped last year tagged a message about a desk lamp from a completely different shop as desk_pro with a probability of 0.91. The model was not broken. I had asked "which of these three products is this about?" and it answered that question correctly — the lamp really is closest to the desk product. The question I actually needed answering was "is this about any of our products at all?", and the Jev companion Noul pattern is how you ask both at once.
Why a Choice question can never say no#
A Jev Choice answer carries three fields, and all three describe the same thing: the comparison between the options you supplied.
{
"model": "jev-1.14.0",
"answers": {
"product": {
"type": "choice",
"choice": "desk_pro",
"probabilities": {
"desk_pro": 0.912,
"chair_ergo": 0.071,
"monitor_arm": 0.017
},
"confidence": 0.883
}
},
"usage": { "input_tokens": 104, "output_tokens": 9 }
}
The probabilities map sums to 1.0 across the options you declared. It tells you which option is most likely given that one of them is the answer. It cannot tell you that the premise was false, because nothing in the request ever expressed the premise as something that could be false.
This is why thresholding the Choice does not rescue you. A 0.91 on an off-topic message and a 0.91 on a genuine desk_pro enquiry are the same number. The axis you need — does this fit the world at all — is not in the response.
Why an "other" option makes it worse#
The usual fix from the LLM-classification playbook is to add other to the enum. It is a trap for two reasons.
It competes. other now sits in the same probability simplex as your real options, so every point of mass it takes comes off a genuine category. Slightly unusual phrasing about a real product starts landing in other because the distribution is zero-sum.
It has no criteria of its own. What is other? You end up writing "anything not covered by the options above", which is a definition by negation — and TypeSafe documents that indirection degrades accuracy. The model is being asked to reason about the absence of a match while simultaneously ranking matches.
Two questions, two axes. That is the whole idea.
The Jev companion Noul pattern#
A Noul is Jev's third primitive: a statement, answered with a single float between 0 and 1 representing the probability the statement is true. Put one alongside the Choice in the same questions object and both are evaluated in the same parallel pass — same state, one round trip, one extra question's worth of tokens. The economics are the same ones I worked through in batching Jev questions into one request: the state dominates the bill, so the marginal question is nearly free and the marginal request is not.
Define the catalogue once, in a backed enum, so the Choice criteria and the Noul criteria can never drift apart:
<?php
declare(strict_types=1);
namespace App\Support\Inbox;
enum Product: string
{
case DeskPro = 'desk_pro';
case ChairErgo = 'chair_ergo';
case MonitorArm = 'monitor_arm';
public function description(): string
{
return match ($this) {
self::DeskPro => 'The Desk Pro sit-stand desk: height memory, cable tray, desktop finishes.',
self::ChairErgo => 'The Ergo task chair: lumbar support, armrests, casters, gas lift.',
self::MonitorArm => 'The single and dual monitor arms: VESA plates, clamps, grommets.',
};
}
/** @return array<string, string> The Choice criteria map Jev sees. */
public static function criteria(): array
{
return array_reduce(
self::cases(),
static fn (array $carry, self $case): array => [
...$carry,
$case->value => $case->description(),
],
[],
);
}
}
Now the request. Both questions see the same state, so the message is sent once:
{
"state": {
"message": "Bought a desk lamp from you lot in March and the switch has gone. Warranty?",
"channel": "email"
},
"model": "jev-1.14.0",
"questions": {
"product": {
"type": "choice",
"instructions": "Which product is this message about?",
"criteria": {
"desk_pro": "The Desk Pro sit-stand desk: height memory, cable tray, desktop finishes.",
"chair_ergo": "The Ergo task chair: lumbar support, armrests, casters, gas lift.",
"monitor_arm": "The single and dual monitor arms: VESA plates, clamps, grommets."
}
},
"about_our_product": {
"type": "noul",
"instructions": "This message is about one of the three products listed in the product question's criteria.",
"criteria": {
"true": "Names, describes or asks about the Desk Pro desk, the Ergo chair or a monitor arm, including parts, warranty, delivery and returns for those items.",
"false": "A product we do not sell (lamps, keyboards, laptops), another vendor's product, a service enquiry, a recruiter or sales email, spam, or a message with no product in it at all."
}
}
}
}
And the answer, on that same lamp message:
{
"model": "jev-1.14.0",
"answers": {
"product": {
"type": "choice",
"choice": "desk_pro",
"probabilities": { "desk_pro": 0.912, "chair_ergo": 0.071, "monitor_arm": 0.017 },
"confidence": 0.883
},
"about_our_product": { "type": "noul", "noul": 0.02 }
},
"usage": { "input_tokens": 141, "output_tokens": 13 }
}
The Choice is unchanged and still confident. The Noul is the field that caught it.
Writing the Noul's false criteria#
criteria on a Noul is optional and takes exactly two keys, true and false. It is also where all of the accuracy lives. Compare:
// Vague. "Relevant" is doing an enormous amount of unspecified work.
new Noul('This message is relevant to our products.');
// Specific. The false list is the one that pays.
new Noul(
instructions: 'This message is about one of the three products listed in the product question\'s criteria.',
criteria: [
'true' => 'Names, describes or asks about the Desk Pro desk, the Ergo chair or a monitor arm, including parts, warranty, delivery and returns for those items.',
'false' => 'A product we do not sell (lamps, keyboards, laptops), another vendor\'s product, a service enquiry, a recruiter or sales email, spam, or a message with no product in it at all.',
],
);
Write the false list from your own misroutes, not from your imagination. Mine came out of an afternoon reading the general inbox: recruiter mail, invoice chasers, a furniture brand with a similar name, and people asking whether we do office fit-outs. Four categories, four clauses, and the false-positive rate dropped off a cliff.
Phrase the statement so that high always means yes. "This is not off topic" is two negations deep by the time your match block reads it, and you will get the band boundaries wrong at 11pm.
Read the Noul first in code#
The answers come back as a flat map, so decode them into something typed before anything branches on them. The Noul, Choice and Jev classes here are the thin wrappers from batching Jev questions into one request — there is still no official PHP SDK, only Python and JavaScript, so this is about thirty lines of Http::withToken().
<?php
declare(strict_types=1);
namespace App\Support\Inbox;
final readonly class InboxVerdict
{
private function __construct(
public float $aboutOurProduct,
public ?Product $product,
public float $productConfidence,
/** @var array<string, float> */
public array $probabilities,
public string $model,
) {}
/** @param array<string, mixed> $payload The decoded System One response. */
public static function fromResponse(array $payload): self
{
$choice = $payload['answers']['product'];
return new self(
// A Noul answer has only a `noul` key — there is no confidence on it.
aboutOurProduct: (float) $payload['answers']['about_our_product']['noul'],
product: Product::tryFrom($choice['choice']),
productConfidence: (float) $choice['confidence'],
probabilities: $choice['probabilities'],
// Pin the versioned model on the record, never the alias.
model: $payload['model'],
);
}
}
Thresholds go in config, not in the prompt and not inline:
<?php
// config/jev.php
return [
'model' => env('JEV_MODEL', 'jev-1.14.0'),
'inbox' => [
// Below this, the Choice is meaningless regardless of its probability.
'noul_low' => (float) env('JEV_INBOX_NOUL_LOW', 0.25),
// At or above this, trust the Noul and look at the Choice.
'noul_high' => (float) env('JEV_INBOX_NOUL_HIGH', 0.80),
// The Choice's own gate, once the Noul has cleared.
'choice_min' => (float) env('JEV_INBOX_CHOICE_MIN', 0.70),
],
];
Then the routing decision is a plain class with no network in it, which means you can unit test every band:
<?php
declare(strict_types=1);
namespace App\Support\Inbox;
enum Queue: string
{
case General = 'general';
case HumanReview = 'human_review';
case ProductTeam = 'product_team';
}
final readonly class RouteEnquiry
{
public function __construct(
private float $noulLow,
private float $noulHigh,
private float $choiceMin,
) {}
public function decide(InboxVerdict $verdict): Queue
{
return match (true) {
// Noul first, always. A low Noul voids the Choice entirely.
$verdict->aboutOurProduct < $this->noulLow => Queue::General,
$verdict->aboutOurProduct < $this->noulHigh => Queue::HumanReview,
$verdict->product === null => Queue::HumanReview,
$verdict->productConfidence < $this->choiceMin => Queue::HumanReview,
default => Queue::ProductTeam,
};
}
}
Three bands, one expression, and the order of the arms is the policy. The same shape drives confidence-gated ticket triage; the only difference is which answer gets read first.
Testing the Jev companion Noul with Http::fake()#
Three canned responses, three assertions. This is the part that keeps the pattern maintainable when someone widens the catalogue in six months.
<?php
use App\Support\Inbox\InboxVerdict;
use App\Support\Inbox\Queue;
use App\Support\Inbox\RouteEnquiry;
use Illuminate\Support\Facades\Http;
/** @param array{noul: float, choice: string, confidence: float} $answer */
function jevResponse(array $answer): array
{
return [
'model' => 'jev-1.14.0',
'answers' => [
'product' => [
'type' => 'choice',
'choice' => $answer['choice'],
'probabilities' => [$answer['choice'] => $answer['confidence']],
'confidence' => $answer['confidence'],
],
'about_our_product' => ['type' => 'noul', 'noul' => $answer['noul']],
],
];
}
beforeEach(function (): void {
$this->router = new RouteEnquiry(noulLow: 0.25, noulHigh: 0.80, choiceMin: 0.70);
});
it('sends an off-topic message to the general inbox despite a confident choice', function (): void {
Http::fake(['api.typesafe.ai/*' => Http::response(
jevResponse(['noul' => 0.02, 'choice' => 'desk_pro', 'confidence' => 0.883])
)]);
$verdict = InboxVerdict::fromResponse(
Http::post('https://api.typesafe.ai/v1/systemone')->json()
);
expect($this->router->decide($verdict))->toBe(Queue::General);
});
it('sends a borderline noul to human review', function (): void {
Http::fake(['api.typesafe.ai/*' => Http::response(
jevResponse(['noul' => 0.54, 'choice' => 'chair_ergo', 'confidence' => 0.91])
)]);
$verdict = InboxVerdict::fromResponse(
Http::post('https://api.typesafe.ai/v1/systemone')->json()
);
expect($this->router->decide($verdict))->toBe(Queue::HumanReview);
});
it('sends a confident on-topic enquiry to the product team', function (): void {
Http::fake(['api.typesafe.ai/*' => Http::response(
jevResponse(['noul' => 0.97, 'choice' => 'monitor_arm', 'confidence' => 0.88])
)]);
$verdict = InboxVerdict::fromResponse(
Http::post('https://api.typesafe.ai/v1/systemone')->json()
);
expect($this->router->decide($verdict))->toBe(Queue::ProductTeam);
});
Add a fourth test the day you widen the catalogue: a high Noul with a Choice confidence of 0.4, asserting human review. That is the case a new option quietly creates.
Gotchas and Edge Cases#
A Noul answer has no confidence field. This is the single most common error in third-party writing about Jev. The answer object contains noul and nothing else — no confidence, no probabilities, no legend. Reading $answers['about_our_product']['confidence'] gets you null and a routing bug that only shows up on the branch you tested least. The probability is the confidence signal: 0.97 and 0.03 are both confident; 0.5 is the model saying it cannot tell.
0.5 is not yes. A Noul of exactly 0.5 rounds up in anyone's head and in some casting. It is the single most uncertain answer the model can give you, and it belongs in the dead band. Never derive your bands from round().
Reading the Choice first. If your code pulls choice before it has looked at the Noul, you have written the bug this entire pattern exists to prevent — just with extra JSON. Make the Noul the first arm of the match (true).
Thresholds in two places. A number in the job and the same number in a test fixture is a number that will disagree with itself. Resolve config('jev.inbox.noul_low') at the call site and inject it.
jev-latest moves underneath you. The alias was repointed from 1.13.0 to 1.14.0 a week after that release. Pin jev-1.14.0 and store it on the record alongside the two answers, or every threshold you tuned drifts on a day you were not deploying.
Questions do not see each other. The Noul cannot read the Choice's answer and no arithmetic relationship between the two is guaranteed. They are two independent judgements about the same state, which is exactly why the pattern works — and why you must not write a Noul whose statement refers to "the answer above".
Wrapping Up#
Add the Noul to the request you already send, store both answers with the versioned model string, and make the Noul the first arm of your routing match. It is a twenty-minute change to an existing classifier, and it converts a confidently wrong label into a correctly routed message.
From here, the same two-axis thinking composes: weighted composite scoring when several answers have to be combined into one verdict, and hierarchical classification down a category tree when the catalogue outgrows a single flat Choice. If the classifier sits downstream of an LLM, verifying that output with Jev is the other half of the same job.
FAQ#
Why does a Choice question always return an option?
Because the question you asked is "which of these?", not "does any of these fit?". The probabilities map is distributed across the options you supplied and sums to 1.0, so it ranks your options against each other rather than against the world. An input that matches none of them still has a closest option, and that option is a correct answer to the question as posed.
What is a companion Noul in TypeSafe's System One API?
It is a second question sent in the same request as a Choice, asking whether any of the Choice's options applies at all. A Noul is Jev's boolean primitive: a statement plus optional true and false criteria, answered with a single float between 0 and 1. Because all questions in a request are evaluated in one parallel pass against the same state, the companion costs one extra question rather than a second round trip.
How do I handle none-of-the-above in AI classification?
Put the "whether" on a separate axis from the "which". Adding an other option to the enum does not work, because it competes for probability mass with your real categories and has no criteria of its own beyond a definition by negation. A companion Noul sits outside that distribution, so a confident Choice and a near-zero Noul can coexist — which is exactly what an off-topic input looks like.
Can I send a Choice and a Noul in the same Jev request?
Yes, and you should. All question types go to the same POST https://api.typesafe.ai/v1/systemone endpoint in one body of state, model and questions, and you can mix types freely inside questions. Since Jev bills input tokens and the state usually dominates them, two questions about one message cost roughly what one costs, while two separate requests pay for the state twice.
What thresholds should I use for Noul answers?
There is no universal number — pick yours from your own labelled examples and keep a dead band in the middle where nothing happens automatically. I start at 0.25 and 0.80 for inbox routing and tune from there, with a tighter gate on any consequence that is hard to undo. Store every Noul value you see for a few weeks before you commit to the boundaries.