A lead came out of our scoring pipeline at 62 and the sales lead asked the only question that matters: why 62? The honest answer was "the model said so". Worse, when the business decided that budget mattered twice as much as urgency that quarter, the only lever was the prompt — and moving it moved every other score at the same time. The fix is to stop asking for a verdict. Ask four small questions the model can actually answer, and do the arithmetic yourself.
Break the judgement into atomic scores#
Start by writing down the factors a human would weigh, then check each one can be answered from the state alone without reference to the others. "Is this a good lead?" is not answerable. "How concretely is a budget indicated?" is. For a sales enquiry I land on four: budget signal, urgency, decision-making authority, and fit with what we actually do.
The test for a good decomposition is independence. If answering one question requires knowing the answer to another, they're one question wearing two hats, and Jev will give you noise on both — questions in a request are evaluated in parallel against the same state and cannot see each other.
<?php
declare(strict_types=1);
namespace App\Support\Scoring;
enum ScoreComponent: string
{
case Budget = 'budget_signal';
case Urgency = 'urgency';
case Authority = 'authority';
case Fit = 'fit';
public function label(): string
{
return match ($this) {
self::Budget => 'Budget signal',
self::Urgency => 'Urgency',
self::Authority => 'Decision-making authority',
self::Fit => 'Fit with our services',
};
}
}
The enum is not decoration. It becomes the question ids, the config keys, the breakdown keys and the column labels in the UI, which means a typo in any one of those is a PHP error rather than a silently missing component.
Write Score legends that define each level#
A Score places the state on an ordered rubric and returns a fractional position: levels are numbered by their index in criteria starting at zero, so four levels give you a number between 0.0 and 3.0. Jev evaluates each level independently and never sees its number or its neighbours, which is why the level text is the whole specification.
Degrees are useless. "Low", "medium", "high" gives the model nothing to match against, and the Score docs show numeric-only criteria collapsing confidence on inputs that descriptive levels place cleanly. Write situations instead:
// Vague — the model is guessing what "medium" means to you.
'criteria' => ['Low budget', 'Medium budget', 'High budget'],
// Specific — every level describes something observable in the state.
'criteria' => [
'No mention of money, cost or budget anywhere in the enquiry.',
'Costs raised in the abstract: asking for a price list, day rate, or "rough idea".',
'A range or ballpark named, or an existing spend referenced.',
'A specific approved budget stated, with a figure.',
],
That lesson came out of building a Jev-backed spam and toxicity validation rule, where numeric level labels flattened confidence on exactly the submissions that needed a confident answer. Keep each component to one dimension, keep the levels mutually exclusive, and read them back as a list — if two levels could both be true of the same enquiry, the rubric is wrong.
<?php
declare(strict_types=1);
namespace App\Support\Scoring;
use App\Support\Jev\QuestionSet;
use App\Support\Jev\Score;
final class LeadQuestions
{
public static function all(): QuestionSet
{
return QuestionSet::make()
->add(ScoreComponent::Budget->value, new Score(
instructions: 'How concretely does `enquiry.body` indicate a budget?',
criteria: [
'No mention of money, cost or budget anywhere in the enquiry.',
'Costs raised in the abstract: a price list, day rate, or "rough idea".',
'A range or ballpark named, or an existing spend referenced.',
'A specific approved budget stated, with a figure.',
],
))
->add(ScoreComponent::Urgency->value, new Score(
instructions: 'How time-pressured is the sender of `enquiry.body`?',
criteria: [
'No timeframe mentioned; exploratory.',
'A soft timeframe: "later this year", "at some point".',
'A named quarter, month or event the work must land before.',
'A hard date within weeks, or work already late.',
],
))
->add(ScoreComponent::Authority->value, new Score(
instructions: 'How much authority does the sender of `enquiry.body` appear to have over this decision?',
criteria: [
'Researching on behalf of someone else, or role unknown.',
'Involved in the decision but says it needs sign-off.',
'Speaks as the person choosing a supplier.',
'Founder, director or budget holder writing directly.',
],
))
->add(ScoreComponent::Fit->value, new Score(
instructions: 'How well does the work described in `enquiry.body` match a Laravel application build or rescue?',
criteria: [
'Unrelated to software delivery.',
'Software, but outside our stack: mobile-only, data science, hardware.',
'Web application work we could do, stack unstated.',
'Explicitly Laravel, PHP or an existing app of ours.',
],
));
}
}
One escalation is worth knowing about before you start rewriting legends by feel. If the model keeps landing between two neighbouring levels on enquiries you think are obvious, each level in criteria can be an object rather than a string — a what field plus an examples array — using the same field names on every level. TypeSafe's own figures show a matching example pulling a split 1.43/0.35 answer to 1.03/0.96, and an unrelated example changing nothing at all. Pick examples whose correct level you already know, then re-test on inputs you didn't tune against.
Ask every component score in one request#
All four questions go in a single POST /v1/systemone with the state sent once. This is the point of the pattern working out cheaper than the one-verdict prompt it replaces: Jev bills input tokens and the state dominates them, so four questions about one enquiry cost roughly what one question costs. The mechanics — the QuestionSet, the thin Http::withToken() client, the Answers wrapper — are the ones I built in batching Jev questions into one request, and nothing here changes them.
<?php
declare(strict_types=1);
namespace App\Support\Scoring;
use App\Models\Enquiry;
use App\Support\Jev\Jev;
final readonly class LeadScorer
{
public function __construct(private Jev $jev, private CompositeScorer $composite) {}
public function score(Enquiry $enquiry): CompositeScore
{
$answers = $this->jev->ask(
EnquiryState::for($enquiry),
LeadQuestions::all(),
);
// The API call ends here. Everything below this line is pure PHP.
return $this->composite->combine($answers);
}
}
Draw that boundary deliberately. One method does I/O, everything after it is arithmetic you can test without an API key, and the thing a customer will ask you to justify lives entirely on the testable side.
Combine the scores with weights from config#
The weights belong in config/scoring.php, never in the question text. A weight change is then a one-line diff, reviewable, deployable and testable, and it moves nothing about how the model reads the enquiry. Version the config too — comparing a total from March against one from September is meaningless unless you know both were scored under the same weights.
<?php
declare(strict_types=1);
return [
// Bump this whenever the weights below change. Stored with every score.
'version' => 3,
// Weights are relative, not percentages — the scorer normalises them.
'weights' => [
'budget_signal' => 4,
'urgency' => 2,
'authority' => 3,
'fit' => 3,
],
// Levels per component, for normalising each raw score to 0..1.
'levels' => 4,
// Below this, a component is too uncertain to count on.
'confidence_floor' => 0.55,
];
Now the combination itself. Each raw score is normalised to 0–1 by dividing by levels - 1, multiplied by its share of the total weight, and summed. The result is a readonly value object carrying every component's contribution — the readonly class as a value object pattern, so a score can't be quietly mutated between calculation and persistence.
<?php
declare(strict_types=1);
namespace App\Support\Scoring;
use App\Support\Jev\Answers;
final readonly class CompositeScorer
{
/** @param array{version: int, weights: array<string, int|float>, levels: int, confidence_floor: float} $config */
public function __construct(private array $config) {}
public function combine(Answers $answers): CompositeScore
{
$weights = $this->config['weights'];
$totalWeight = array_sum($weights);
$maxLevel = $this->config['levels'] - 1;
$components = [];
foreach (ScoreComponent::cases() as $component) {
$raw = $answers->score($component->value);
$confidence = $answers->confidence($component->value);
$share = $weights[$component->value] / $totalWeight;
$components[] = new ScoreContribution(
component: $component,
raw: $raw,
confidence: $confidence,
weight: $share,
// Normalise to 0..1, then take this component's weighted share of 100.
contribution: ($raw / $maxLevel) * $share * 100,
);
}
return new CompositeScore(
components: $components,
weightsVersion: $this->config['version'],
confidenceFloor: $this->config['confidence_floor'],
);
}
}
CompositeScore holds the parts and derives the total, rather than being handed a number:
<?php
declare(strict_types=1);
namespace App\Support\Scoring;
final readonly class CompositeScore
{
/** @param list<ScoreContribution> $components */
public function __construct(
public array $components,
public int $weightsVersion,
private float $confidenceFloor,
) {}
public function total(): float
{
return round(array_sum(array_map(
static fn (ScoreContribution $c): float => $c->contribution,
$this->components,
)), 1);
}
/** @return list<ScoreContribution> */
public function uncertain(): array
{
return array_values(array_filter(
$this->components,
fn (ScoreContribution $c): bool => $c->confidence < $this->confidenceFloor,
));
}
public function needsReview(): bool
{
return $this->uncertain() !== [];
}
}
Doubling the weight on budget is now 'budget_signal' => 8, a bumped version, and a test asserting the new total. No prompt touched, and every other component moves in the way arithmetic says it should.
Handle low-confidence components explicitly#
Confidence is a second axis, not a tiebreaker. Jev derives it from how the probability spreads across the levels: 1.0 when it all lands on one, lower the more it smears. A budget score of 1.5 at confidence 0.9 means "somewhere between abstract and ballpark" — useful. The same 1.5 at confidence 0.2 means "I cannot place this", and averaging it into a total launders a shrug into a decimal.
So don't. Let the composite carry the uncertainty up to the caller and make the decision there, the same confidence-band approach as confidence-gated ticket triage with Jev.
$score = $leadScorer->score($enquiry);
$enquiry->update([
'lead_score' => $score->total(),
'needs_review' => $score->needsReview(),
]);
// A 78 that depends on a component Jev couldn't place is not a 78.
if ($score->needsReview()) {
SalesReviewQueue::push($enquiry, reason: $score->uncertain());
}
Resist the urge to drop an uncertain component and re-weight the rest. That silently changes what the number means — a lead scored on three factors is not comparable with one scored on four, and nothing in the stored total would tell you which you were looking at later.
Persist the breakdown as an audit trail#
Store the parts. A total alone cannot be explained, re-derived, or audited after a weight change, and "why 62?" is a question that arrives months later. A JSON column cast to the value object gives you the whole decision back.
php artisan make:migration add_score_breakdown_to_enquiries_table --no-interaction
Schema::table('enquiries', function (Blueprint $table): void {
$table->decimal('lead_score', 5, 1)->nullable()->index();
$table->json('score_breakdown')->nullable();
$table->unsignedSmallInteger('score_weights_version')->nullable();
$table->string('score_model')->nullable(); // e.g. jev-1.13.0
$table->boolean('needs_review')->default(false);
$table->timestamp('scored_at')->nullable();
});
A custom cast keeps the hydration in one place rather than scattering json_decode() through the app. Give CompositeScore and ScoreContribution a toArray() and a static fromArray() — plain field mapping, no cleverness, because this is the code that has to still work when you read the row back in two years:
<?php
declare(strict_types=1);
namespace App\Casts;
use App\Support\Scoring\CompositeScore;
use Illuminate\Contracts\Database\Eloquent\CastsAttributes;
use Illuminate\Database\Eloquent\Model;
/** @implements CastsAttributes<CompositeScore|null, CompositeScore|null> */
final class AsCompositeScore implements CastsAttributes
{
public function get(Model $model, string $key, mixed $value, array $attributes): ?CompositeScore
{
return $value === null
? null
: CompositeScore::fromArray(json_decode($value, true, flags: JSON_THROW_ON_ERROR));
}
public function set(Model $model, string $key, mixed $value, array $attributes): ?string
{
return $value instanceof CompositeScore
? json_encode($value->toArray(), JSON_THROW_ON_ERROR)
: null;
}
}
Register it on the model in the casts() method, alongside the rest:
protected function casts(): array
{
return [
'score_breakdown' => AsCompositeScore::class,
'needs_review' => 'boolean',
'scored_at' => 'immutable_datetime',
];
}
Write the model string from the response as well. When TypeSafe ships a new Jev version your scores shift slightly, and the only way to see that in your data is to have recorded which model produced each one — the same reason I log usage per call when tracking token usage and cost.
Render the breakdown where people ask about it#
The audit trail is only worth the column if sales can see it without asking an engineer. A table of component, level, weight and contribution, sorted by contribution, answers "why 62?" on its own.
<table class="w-full text-sm">
<tbody>
@foreach ($enquiry->score_breakdown->components as $component)
<tr @class(['opacity-60' => $component->confidence < 0.55])>
<td>{{ $component->component->label() }}</td>
<td>{{ $component->levelLabel() }}</td>
<td class="tabular-nums">{{ number_format($component->weight * 100) }}%</td>
<td class="tabular-nums">+{{ number_format($component->contribution, 1) }}</td>
<td>
@if ($component->confidence < 0.55)
<span title="Jev was unsure here">uncertain</span>
@endif
</td>
</tr>
@endforeach
</tbody>
</table>
Show the level's own legend text in levelLabel(), not the number. "A range or ballpark named" is the sentence that ends the argument; "2.0" restarts it.
Test the weighting maths without the API#
The arithmetic is pure PHP, so test it natively with a hand-built Answers object and no HTTP at all. These are the tests that stop a weight change breaking a threshold somewhere downstream.
<?php
declare(strict_types=1);
use App\Support\Jev\Answers;
use App\Support\Scoring\CompositeScorer;
function answers(array $scores, float $confidence = 0.9): Answers
{
$payload = [];
foreach ($scores as $id => $score) {
$payload[$id] = ['type' => 'score', 'score' => $score, 'confidence' => $confidence];
}
return new Answers($payload, ['input_tokens' => 0, 'output_tokens' => 0]);
}
it('scores a perfect lead at 100', function (): void {
$score = app(CompositeScorer::class)->combine(answers([
'budget_signal' => 3.0, 'urgency' => 3.0, 'authority' => 3.0, 'fit' => 3.0,
]));
expect($score->total())->toBe(100.0);
});
it('weights budget above urgency', function (): void {
$scorer = app(CompositeScorer::class);
$budgetOnly = $scorer->combine(answers(
['budget_signal' => 3.0, 'urgency' => 0.0, 'authority' => 0.0, 'fit' => 0.0]
));
$urgencyOnly = $scorer->combine(answers(
['budget_signal' => 0.0, 'urgency' => 3.0, 'authority' => 0.0, 'fit' => 0.0]
));
// 4/12 of the weight versus 2/12 — exactly double, by config.
expect($budgetOnly->total())->toBe(33.3)
->and($urgencyOnly->total())->toBe(16.7);
});
it('flags a total built on an uncertain component', function (): void {
$scorer = app(CompositeScorer::class);
$score = $scorer->combine(new Answers([
'budget_signal' => ['type' => 'score', 'score' => 3.0, 'confidence' => 0.2],
'urgency' => ['type' => 'score', 'score' => 2.0, 'confidence' => 0.95],
'authority' => ['type' => 'score', 'score' => 2.0, 'confidence' => 0.95],
'fit' => ['type' => 'score', 'score' => 3.0, 'confidence' => 0.99],
], ['input_tokens' => 0, 'output_tokens' => 0]));
expect($score->needsReview())->toBeTrue()
->and($score->uncertain())->toHaveCount(1);
});
A single Http::fake() test covering the full LeadScorer path is still worth having, to catch a question id that stops matching a config key. Beyond that, a custom expectation like expect($score)->toScoreBetween(60, 80) reads better across a dataset of real enquiries — see building custom Pest expectations for the pattern.
Gotchas and Edge Cases#
Never compare totals across weight versions. A 71 scored under version 2 and a 68 under version 3 are measurements of different things. Store score_weights_version, filter on it in any report, and backfill by re-combining from the stored breakdown — which you can do without calling the API at all, because you kept the raw component scores.
Scores are not probabilities. A composite of 78 does not mean a 78% chance of closing. It's a weighted position on rubrics you wrote. If you want a probability, that's a model fitted on your own outcome data, with this score as one input.
A score of 1.5 may not mean level 1.5. Jev returns the expected value across the distribution, so 1.5 can mean "firmly between levels 1 and 2" or "half the probability on 0 and half on 3". Those are very different enquiries. Confidence is what separates them, which is the real reason the floor exists.
Don't let the weights become meaningless. Normalising by array_sum($weights) means relative values are all that matter, which is a feature — but it also means nobody notices when a fifth component is added and the original four quietly shrink. Assert the component list against ScoreComponent::cases() in a test.
Watch the level count assumption. The scorer divides by levels - 1 from a single config value, so giving one component five levels and the rest four silently skews it. Either keep every rubric the same depth or store the level count per component.
Wrapping Up#
Decompose first, write the legends carefully, keep the weights in config, and store the breakdown. The arithmetic is the easy part — the value is that every number you produce can be re-derived and defended a year later, by someone who wasn't there.
If you're applying the same idea to text a model just generated rather than to inbound enquiries, verifying LLM output with Jev covers the guardrail shape, and batching Jev questions into one request is the piece to read if your component questions are still going out as separate calls.
FAQ#
How do you make AI scoring explainable?
Stop asking for the final number. Decompose the judgement into several narrow questions a model can answer independently, combine them with weights held in your own code, and store every component alongside the total. The explanation then writes itself: this lead scored 62 because budget contributed 11 of a possible 33 and fit contributed 25 of a possible 25.
Can AI do lead scoring?
It can do the parts of lead scoring that require reading, which is judging what an enquiry says about budget, urgency, authority and fit. It should not own how much each of those is worth to your business — that is a commercial decision that changes quarterly and belongs in config where it can be reviewed and tested. Split the work that way and the model does what it is good at.
What is a weighted scoring model?
It is a scoring method where each factor contributes a defined share of the total rather than counting equally. Each factor is scored on its own scale, normalised to a common range, multiplied by its weight, and summed. The weights encode priorities, so changing one changes the ranking without changing how any individual factor is assessed.
How do Jev Score levels work?
A Score question takes an ordered list of level descriptions in criteria and returns a fractional position on that list, indexed from zero — four levels produce a value between 0.0 and 3.0. The model evaluates each level independently against the state and never sees the level numbers, so descriptive levels that name an observable situation score far more confidently than labels like "low" or "high".
How do I store an AI score breakdown in Eloquent?
Add a json column for the breakdown and a custom cast implementing CastsAttributes that hydrates it into a value object on read and encodes it on write. Register the cast in the model's casts() method. Store the weights version and the model identifier in their own columns so a stored total can be re-derived and compared honestly later.
What do I do when one input score has low confidence?
Flag the composite rather than hiding the uncertainty inside it. Keep the component in the calculation so the total still means what it claims to mean, set a needs_review flag when any component falls below your confidence floor, and route those records to a human. Dropping the uncertain component and re-weighting the rest produces a number that silently measures something different.