Your Search Engine Found It at Position 14: Re-Ranking Scout Results with Jev

A client's docs search was "useless". It wasn't. Every time I checked a failing query, the document the user wanted was in the results — at position 14, under thirteen pages that happened to share more words with the query. Scout had done its job. The ordering hadn't.

That is the normal failure mode of lexical search, and the normal advice is to rip out the index and build a vector pipeline. You rarely need to. The right answer is already on the shortlist; what you need is a second pass that reads the shortlist and puts it in a better order.

Measure where the right result actually lands#

Before changing anything, find out how bad the ordering is, because "search is useless" is not a number. Take 30 or 40 real queries from your logs, write down the one document each should return, and record the position your current search gives it. Two figures matter: how often the right document is anywhere in the top 30 (recall — this is what you must not break), and how often it is at position 1 (precision — this is what you are about to fix).

<?php

declare(strict_types=1);

namespace App\Console\Commands;

use App\Models\Article;
use Illuminate\Console\Command;

final class MeasureSearchQuality extends Command
{
    protected $signature = 'search:measure {--rerank}';

    /** @var array<string, int> Query => the id of the article that should win. */
    private const LABELLED = [
        'how do I rotate the api token without logging everyone out' => 412,
        'invoice shows the wrong vat for irish customers' => 87,
        // ...30 more, pulled from the search log and labelled by hand.
    ];

    public function handle(): int
    {
        $ranks = [];

        foreach (self::LABELLED as $query => $expectedId) {
            $ids = Article::search($query)->take(30)->keys()->all();
            $position = array_search($expectedId, $ids, strict: true);

            // false means the index never surfaced it — a recall problem, not an ordering one.
            $ranks[$query] = $position === false ? null : $position + 1;
        }

        $total = count($ranks);
        $top1 = count(array_filter($ranks, fn (?int $rank): bool => $rank === 1));
        $top10 = count(array_filter($ranks, fn (?int $rank): bool => $rank !== null && $rank <= 10));

        $this->table(['metric', 'share'], [
            ['in top 30', sprintf('%d%%', 100 * count(array_filter($ranks)) / $total)],
            ['at rank 1', sprintf('%d%%', 100 * $top1 / $total)],
            ['in top 10', sprintf('%d%%', 100 * $top10 / $total)],
        ]);

        return self::SUCCESS;
    }
}

Run it and keep the output. On the client's set, the right document was in the top 30 for 94% of queries and first for 21%. That gap is the entire business case: the index is fine, the order is not. TypeSafe's own re-ranking cookbook measures the same shape on 3,565 CLERC court-opinion passages — the gold passage made the BM25 top 30 for 100% of its 40 queries and landed first for 5%.

Pull a wider shortlist from Scout#

Re-ranking can only reorder what the index hands it. If your search page shows 10 results and you ask Scout for 10, re-ranking can shuffle those 10 and nothing else — the document at position 14 stays invisible. So retrieve wider than you display: fetch 20–30 candidates, re-rank all of them, show the top 5 or 10.

$candidates = Article::search($query)
    ->take(30)                       // retrieve wide, display narrow
    ->query(fn ($builder) => $builder->select(['id', 'title', 'slug', 'excerpt']))
    ->get();

Thirty is where I start. Below about 15 there is not enough room for a result to move far enough to matter; above about 50 you are paying to score candidates the index itself ranked as barely relevant, and the gain flattens. The fast-search layer underneath is a free choice — this is the same Scout setup I wrote about in typo-tolerant search with Scout and Typesense, and re-ranking never sees the driver, only the rows it returns.

Score every candidate with one Noul each#

This is where the brief I started from was wrong, so it is worth being precise. A Jev Score returns a position on a spectrum you define with ordered level descriptions, and it looks like the obvious fit. It isn't. What a sort needs is a number that is comparable across candidates and finely spaced, and that is what a Noul gives you: the probability that the answer to a yes/no question is yes, from 0 to 1, with no thresholding in between.

Ask one question — does this candidate answer the query? — once per candidate, and sort on the returned probability. TypeSafe's cookbook does exactly this, and the Noul docs are explicit that re-ranking uses the probability itself rather than a threshold.

<?php

declare(strict_types=1);

namespace App\Support\Search;

use App\Support\Jev\Noul;

final readonly class AnswersQuery
{
    private const QUESTION = 'Does `candidate` directly answer the question in the query?';

    /**
     * Structured instructions: the question text is fixed, the candidate changes.
     * The query lives in the state, so it is sent once for the whole request.
     */
    public static function for(string $title, string $excerpt): Noul
    {
        return new Noul(
            instructions: [
                'candidate' => ['title' => $title, 'text' => $excerpt],
                'question' => self::QUESTION,
            ],
            criteria: [
                'true' => 'The candidate contains the specific answer the query asks for, even if it uses different wording.',
                'false' => 'The candidate is on a related topic but does not contain the answer.',
            ],
        );
    }
}

Write the criteria as situations, not degrees. "Contains the specific answer, even in different wording" is something the model can judge; "is highly relevant" is not. And phrase the question so that high means good — invert it and every sortByDesc in your codebase reads backwards.

Send all the candidates in a single request#

The cookbook fires one HTTP call per query-candidate pair — 1,200 calls for 40 queries — and says plainly that a real application would ask several questions about the same state in one call. Do that. Jev charges for the state you send, and in this shape the state is the query: send it 30 times and you pay for it 30 times for no benefit.

So the request is one state holding the query, and 30 questions holding the candidates, one question id per candidate id. This is the same QuestionSet and thin HTTP wrapper I built in batching Jev questions into one request, where TypeSafe's parallel-questions cookbook measures batching at 12.2x cheaper and 10.0x faster with identical answers.

<?php

declare(strict_types=1);

namespace App\Support\Search;

use App\Support\Jev\Jev;
use App\Support\Jev\Noul;
use App\Support\Jev\QuestionSet;
use Illuminate\Support\Collection;

final readonly class JevReranker implements Reranker
{
    public function __construct(private Jev $jev) {}

    /**
     * @param  Collection<int, \App\Models\Article>  $candidates
     * @return array{order: array<int, float>, any_match: float}
     */
    public function score(string $query, Collection $candidates): array
    {
        $questions = $candidates->reduce(
            fn (QuestionSet $set, $article): QuestionSet => $set->add(
                "candidate_{$article->id}",
                AnswersQuery::for($article->title, $article->excerpt),
            ),
            QuestionSet::make(),
        );

        // A coverage question, so an honest empty state beats five irrelevant results.
        $questions = $questions->add('any_match', new Noul(
            instructions: 'Is at least one candidate on the shortlist likely to answer the query?',
        ));

        $answers = $this->jev->ask(['query' => $query], $questions);

        return [
            'order' => $candidates
                ->mapWithKeys(fn ($article): array => [
                    $article->id => $answers->noul("candidate_{$article->id}"),
                ])
                ->all(),
            'any_match' => $answers->noul('any_match'),
        ];
    }
}

Watch the token budget while you are here. Jev 1.13 allows 64k tokens for the whole request and a separate 32k ceiling on the state plus the single longest question — and with 30 candidates in one request, the questions are now the fat part, not the state. Send an excerpt per candidate, never the full document.

Sort the shortlist and gate on coverage#

The sort itself is the boring part, which is the point: because a Noul is a float, there is nothing to parse and no branch for "the model replied with a sentence this time". The coverage Noul is what earns its place here — when nothing on the shortlist answers the query, showing five confident-looking irrelevant results is worse than showing none.

public function search(string $query, int $perPage = 10): Collection
{
    $candidates = Article::search($query)->take(30)->get();

    // Nothing to reorder, or a navigational query that does not deserve the round trip.
    if ($candidates->count() < 2 || str_word_count($query) < 3) {
        return $candidates->take($perPage);
    }

    ['order' => $scores, 'any_match' => $anyMatch] = $this->reranker->score($query, $candidates);

    if ($anyMatch < 0.3) {
        return collect();   // honest empty state: say so rather than pad the page
    }

    return $candidates
        ->sortByDesc(fn ($article): float => $scores[$article->id] ?? 0.0)
        ->values()
        ->take($perPage);
}

The 0.3 threshold is the conservative end of TypeSafe's suggested 0.3/0.7 split: only suppress results when the model is fairly sure nothing fits. I tune that on the labelled set and nowhere else — the same discipline as the confidence gates in confidence-gated intent routing with Jev.

Cache the ranked order on the candidate id set#

The re-rank is stable enough to cache and expensive enough to be worth caching. The trick is the key: hash the query together with the ordered list of candidate ids. Reindex a document, or let a new article into the shortlist, and the id set changes, so the key changes, so the cache invalidates itself. No listeners, no manual flush.

$key = 'rerank:'.hash('xxh128', $query.'|'.$candidates->pluck('id')->implode(','));

$scores = Cache::remember(
    $key,
    now()->addDay(),
    fn (): array => $this->reranker->score($query, $candidates),
);

Repeated searches — the same query from the same page, pagination, a user hitting back — then cost nothing. A day is a long TTL and it is safe precisely because content changes move the key rather than staling the value.

Fall back to the Scout order when TypeSafe is down#

Re-ranking improves an ordering that already works. If the API is slow or down, the correct behaviour is the unranked Scout order, not a 500. Keep the timeout short, retry only the documented back-off cases, and catch everything else.

public function search(string $query, int $perPage = 10): Collection
{
    $candidates = Article::search($query)->take(30)->get();

    try {
        ['order' => $scores] = $this->reranker->score($query, $candidates);
    } catch (\Throwable $exception) {
        report($exception);

        return $candidates->take($perPage);   // degrade to lexical order, invisibly to the user
    }

    return $candidates
        ->sortByDesc(fn ($article): float => $scores[$article->id] ?? 0.0)
        ->take($perPage);
}

Bind a NullReranker that returns the candidates untouched outside production, so nobody burns an API key running the test suite:

$this->app->bind(Reranker::class, fn ($app) => $app->environment('production')
    ? $app->make(JevReranker::class)
    : $app->make(NullReranker::class));

Then assert both paths. The fallback is the test that matters, because it is the one that runs on your worst day:

it('falls back to the scout order when jev fails', function (): void {
    Http::fake(['api.typesafe.ai/*' => Http::response(status: 503)]);

    $results = app(SearchService::class)->search('how do I rotate the api token');

    expect($results->pluck('id')->all())->toBe([12, 48, 7]);   // untouched scout order
});

it('reorders the shortlist by noul', function (): void {
    Http::fake(['api.typesafe.ai/*' => Http::response([
        'answers' => [
            'candidate_12' => ['type' => 'noul', 'noul' => 0.11],
            'candidate_48' => ['type' => 'noul', 'noul' => 0.94],
            'candidate_7' => ['type' => 'noul', 'noul' => 0.62],
            'any_match' => ['type' => 'noul', 'noul' => 0.97],
        ],
        'usage' => ['input_tokens' => 1_400, 'output_tokens' => 90],
    ])]);

    $results = app(SearchService::class)->search('how do I rotate the api token');

    expect($results->pluck('id')->all())->toBe([48, 7, 12]);
});

Re-run the measurement and price it#

Run search:measure --rerank against the same labelled queries and compare the two tables. Report what you actually got, not what the cookbook got. For reference, the CLERC benchmark moved top-1 from 5% to 18%, top-5 from 15% to 35%, and top-10 from 38% to 62% — a large relative gain from a small absolute base, on a deliberately hard legal retrieval task. The client's docs search moved top-1 from 21% to 48%. Your corpus will land somewhere else.

Cost is easy to bound because the whole re-rank is one call. At the jev-1.12 input price the cookbook quotes — $0.042 per million tokens as of September 2026, so check the current rate before you repeat it — a 30-candidate pass carrying roughly 200-token excerpts runs around 7,000 input tokens, a small fraction of a penny per search, before the cache takes most of them away. Keep the labelled set in CI so a prompt tweak that quietly makes ordering worse fails a build; that harness is the one from LLM evals with a golden dataset in Pest.

Gotchas and Edge Cases#

Truncate the candidate text, and be consistent about it. Thirty full articles will blow the 64k request budget. Score a fixed excerpt — the title plus the first 200 words or so, or the matched passage if your driver returns one — and keep that choice in one place, because changing it changes every score.

Do not re-rank short navigational queries. "pricing" and "login" do not need semantic reordering, and if your search box is live-bound you will pay for every keystroke. Gate on word count, on debounce, or both; the str_word_count($query) < 3 check above is the cheap version.

A Noul is not a relevance percentage. 0.74 is the probability that the answer is yes, not "74% relevant". Use it to order and to threshold, never as a figure shown to a user.

Recall is still the index's job. Re-ranking cannot add a document Scout did not return. If your measurement shows the right answer missing from the top 30, fix synonyms, field weights or typo tolerance first — re-ranking a broken shortlist only reorders wrong answers.

Mind where the round trip lands. One extra API call sits in the request lifecycle in front of the user. That is survivable on a results page at a few hundred milliseconds; it is not survivable on a type-ahead dropdown. For a "did you mean" panel, show lexical results immediately and swap in the re-ranked order asynchronously.

Wrapping Up#

Measure first, retrieve 30, score each candidate with one Noul in a single request, sort, cache on the id set, and fall back to the Scout order when anything goes wrong. That is the whole pass, and it leaves your index exactly where it was.

If the ordering needs more than one signal — recency, popularity and relevance together — the next step is composite scoring with weighted Jev decisions. If these results are feeding an LLM rather than a human, score the passages before they reach the model, as in scoring RAG passages before the LLM sees them.

FAQ#

What is search re-ranking and when do I need it?

Re-ranking is a second pass that reorders a shortlist your search index already returned, scoring each candidate against the query directly instead of by word overlap. You need it when your measurements show good recall and bad precision — the right document reliably appears in the top 20 or 30 but rarely first. If the right document is missing from the shortlist entirely, that is a recall problem and re-ranking cannot help with it.

How do I re-rank Laravel Scout results?

Ask Scout for more results than you intend to display, typically 30, then score each one against the query and sort the collection in PHP before returning it. With Jev that means a single request whose state holds the query and whose questions hold one Noul per candidate, followed by a sortByDesc on the returned probabilities. The Scout driver is untouched — re-ranking only ever sees the rows it returned.

Can I use Jev with Laravel without an SDK?

Yes. There is no official PHP SDK for TypeSafe, only Python and JavaScript, so the integration is a thin wrapper around Laravel's HTTP client posting to https://api.typesafe.ai/v1/systemone with state, model and questions. About thirty lines covers it. Because you own the client, the timeout, the retry policy and the fallback to the unranked order become your design decisions rather than a library's defaults.

Is re-ranking better than switching to vector search?

They solve different problems, and re-ranking is far cheaper to try. Vector search improves recall by finding documents that share meaning rather than words; re-ranking improves the ordering within whatever shortlist you already have. If your labelled queries show the right answer is usually in the top 30, re-ranking is the smaller change with the larger payoff. The two also compose — a vector shortlist can be re-ranked in exactly the same way.

How much does re-ranking add to search latency?

One API round trip, because every candidate is scored in the same request rather than one call each. In practice that lands in the low hundreds of milliseconds, which is fine on a results page and too slow for a type-ahead dropdown. Caching the scored order on the query plus the candidate id set removes it entirely for repeated searches, and deferring the pass lets you show lexical results first and reorder afterwards.

How do I cache re-ranked search results in Laravel?

Cache the scores rather than the rendered results, and key them on a hash of the query plus the ordered list of candidate ids. That key changes by itself whenever the index returns a different shortlist, so reindexing or publishing new content invalidates the entry without any listener or manual flush. A long TTL is then safe — a day is reasonable — because changed content can no longer produce a stale hit.