Ask Jev Thirteen Questions at Once: Batching and Speculative Fan-Out in Laravel

A client's enquiry form was running through Jev to decide routing, and the integration had grown an Http::pool() loop: six questions, six concurrent requests, the same 4KB of enquiry context serialised into every one of them. It worked. It also cost six times what it needed to, because Jev charges for the state you send, and that code was sending it six times. The fix is one request with six questions in it — and once you've done that, there's no reason to stop at six.

Build the state once from your domain model#

Everything Jev will judge goes into state, and it goes in exactly once per request. Build it from your models as a structured array rather than a concatenated string: labelled fields give the model something to point at, and your question instructions can then reference a field by name in backticks, the way TypeSafe's State docs describe.

<?php

declare(strict_types=1);

namespace App\Support\Enquiries;

use App\Models\Enquiry;

final readonly class EnquiryState
{
    /**
     * @return array{enquiry: array<string, mixed>, company: array<string, mixed>|null}
     */
    public static function for(Enquiry $enquiry): array
    {
        return [
            'enquiry' => [
                'subject' => $enquiry->subject,
                'body' => $enquiry->body,
                'submitted_at' => $enquiry->created_at->toIso8601String(),
                'source' => $enquiry->source,
            ],
            // Enriched from the CRM where we recognise the domain; null otherwise.
            'company' => $enquiry->company?->only(['name', 'industry', 'employee_count']),
        ];
    }
}

Watch the budget while you're here. Jev 1.13 allows 64k tokens for the whole request — state plus every question combined — and a separate 32k ceiling on the state plus the single longest question. A fat state is what pushes you towards that limit, not the questions.

Define the question set as typed PHP objects#

The payload Jev wants is a map of question id to a typed question object. Hand-building that array works for two questions and rots at ten, so wrap each question type in a small readonly class that knows its own shape. A Question interface with one toArray() method is enough.

<?php

declare(strict_types=1);

namespace App\Support\Jev;

interface Question
{
    /** @return array<string, mixed> */
    public function toArray(): array;
}

final readonly class Noul implements Question
{
    public function __construct(
        private string $instructions,
        /** @var array{true: string, false: string}|null */
        private ?array $criteria = null,
    ) {}

    public function toArray(): array
    {
        return array_filter([
            'type' => 'noul',
            'instructions' => $this->instructions,
            'criteria' => $this->criteria,
        ], static fn (mixed $value): bool => $value !== null);
    }
}

final readonly class Choice implements Question
{
    /** @param array<string, string> $criteria Option key => what that option means. Max 255. */
    public function __construct(
        private string $instructions,
        private array $criteria,
    ) {}

    public function toArray(): array
    {
        return [
            'type' => 'choice',
            'instructions' => $this->instructions,
            'criteria' => $this->criteria,
        ];
    }
}

final readonly class Score implements Question
{
    /** @param list<string> $criteria Ordered level descriptions, lowest first. 2–10 levels. */
    public function __construct(
        private string $instructions,
        private array $criteria,
    ) {}

    public function toArray(): array
    {
        return [
            'type' => 'score',
            'instructions' => $this->instructions,
            'criteria' => $this->criteria,
        ];
    }
}

Describe Score levels as situations, not degrees — "Specific project with a stated budget" beats "high". That lesson came out of building a Jev-backed spam and toxicity validation rule, where numeric level labels flattened confidence badly.

Now the collection itself. Make it immutable, so add() returns a new instance and a question set can be composed in layers — a base set for every enquiry, plus extras for enterprise leads — without one caller mutating another's.

<?php

declare(strict_types=1);

namespace App\Support\Jev;

use Countable;

final readonly class QuestionSet implements Countable
{
    /** @param array<string, Question> $questions */
    private function __construct(private array $questions) {}

    public static function make(): self
    {
        return new self([]);
    }

    public function add(string $id, Question $question): self
    {
        // Spread preserves string keys on PHP 8.1+; a later add() with the same id wins.
        return new self([...$this->questions, $id => $question]);
    }

    /** @return array<string, array<string, mixed>> */
    public function toPayload(): array
    {
        return array_map(
            static fn (Question $question): array => $question->toArray(),
            $this->questions,
        );
    }

    public function count(): int
    {
        return count($this->questions);
    }
}

Adding a question is now one line, which is the whole point. The friction you remove here is what makes the next step psychologically possible.

Send every question in one request#

One POST /v1/systemone carries the state and the entire question map, and answers come back keyed by the ids you chose. Those ids are yours alone — TypeSafe's API reference is explicit that the key is never sent to the model, so write the complete question in instructions even when the id looks self-explanatory.

This is the same thin Http::withToken() client I used for confidence-gated ticket triage with Jev; there's still no official PHP SDK, only Python and JavaScript, so a wrapper of about thirty lines is the integration.

<?php

declare(strict_types=1);

namespace App\Support\Jev;

use Illuminate\Support\Facades\Http;

final readonly class Jev
{
    public function __construct(
        private string $apiKey,
        private string $model = 'jev-latest',
    ) {}

    /** @param string|array<string, mixed> $state */
    public function ask(string|array $state, QuestionSet $questions): Answers
    {
        $response = Http::withToken($this->apiKey)
            ->timeout(20)
            // 429 and 529 are the two documented back-off cases.
            ->retry(3, fn (int $attempt): int => $attempt * 250)
            ->post('https://api.typesafe.ai/v1/systemone', [
                'state' => $state,
                'model' => $this->model,
                'questions' => $questions->toPayload(),
            ])
            ->throw();

        return new Answers($response->json('answers'), $response->json('usage'));
    }
}

The Answers object keeps the response shape in one place instead of scattering ['confidence'] lookups across your application. Noul answers carry a bare probability and no confidence; Choice and Score both carry confidence derived from their distribution.

<?php

declare(strict_types=1);

namespace App\Support\Jev;

final readonly class Answers
{
    /**
     * @param array<string, array<string, mixed>> $answers
     * @param array{input_tokens: int, output_tokens: int} $usage
     */
    public function __construct(private array $answers, private array $usage) {}

    public function noul(string $id): float
    {
        return (float) $this->answers[$id]['noul'];
    }

    public function choice(string $id): string
    {
        return (string) $this->answers[$id]['choice'];
    }

    public function score(string $id): float
    {
        return (float) $this->answers[$id]['score'];
    }

    /** Noul answers have no confidence of their own — only Choice and Score do. */
    public function confidence(string $id): float
    {
        return (float) $this->answers[$id]['confidence'];
    }

    public function inputTokens(): int
    {
        return $this->usage['input_tokens'];
    }
}

Add the speculative questions you might not need#

Here is where the LLM habit has to go. With a text model, every extra question is another prompt and another bill, so you ask only what you need and pay for a second round trip when the first answer surprises you. With Jev the state dominates the request and the questions are a rounding error, so asking a question whose answer only matters sometimes is close to free. TypeSafe calls this speculative fan-out: ask everything any branch of your code might want, then let the code decide what was relevant.

For the enquiry form that means asking about budget and timeline even when the enquiry turns out to be a support request, and asking about the complaint severity even when it turns out to be a new lead.

<?php

declare(strict_types=1);

namespace App\Support\Enquiries;

use App\Support\Jev\Choice;
use App\Support\Jev\Noul;
use App\Support\Jev\QuestionSet;
use App\Support\Jev\Score;

final class EnquiryQuestions
{
    public static function all(): QuestionSet
    {
        return QuestionSet::make()
            // Asked for every enquiry — these drive the top-level branch.
            ->add('category', new Choice(
                instructions: 'What kind of enquiry is `enquiry.body`?',
                criteria: [
                    'new_project' => 'Someone wants work done and is enquiring about engaging us.',
                    'support' => 'An existing client reporting a problem with something we built.',
                    'recruitment' => 'A candidate, agency or supplier pitching themselves.',
                    'other' => 'Anything that fits none of the above.',
                ],
            ))
            ->add('urgency', new Noul(
                instructions: 'Does `enquiry.body` express a deadline or time pressure?',
            ))
            // Speculative: only read when category === new_project.
            ->add('budget_signal', new Score(
                instructions: 'How concretely does `enquiry.body` indicate a budget?',
                criteria: [
                    'No mention of money at all.',
                    'Vague interest in costs or a request for a price list.',
                    'A range, a ballpark figure, or an approved budget named.',
                ],
            ))
            ->add('named_technology', new Noul(
                instructions: 'Does `enquiry.body` name a specific framework, language or platform?',
            ))
            // Speculative: only read when category === support.
            ->add('severity', new Score(
                instructions: 'How badly is the sender blocked, per `enquiry.body`?',
                criteria: [
                    'A question or cosmetic issue; nothing is broken.',
                    'A feature is degraded but there is a workaround.',
                    'Production is down or money is not moving.',
                ],
            ))
            ->add('mentions_data_loss', new Noul(
                instructions: 'Does `enquiry.body` describe lost, corrupted or exposed data?',
            ))
            // Useful whatever the category turns out to be.
            ->add('frustration', new Score(
                instructions: 'How frustrated does the sender of `enquiry.body` appear?',
                criteria: [
                    'Calm and matter-of-fact.',
                    'Frustrated but civil.',
                    'Angry, or threatening to escalate.',
                ],
            ))
            ->add('is_automated', new Noul(
                instructions: 'Is `enquiry.body` a bulk mailshot or automated message rather than a person writing to us?',
            ));
    }
}

Eight questions here; the cookbook's example runs thirteen. Neither number is where it gets expensive.

Branch on the answers in plain PHP#

Every answer is independent — one question's result is never hidden context for another — so the routing is ordinary deterministic code you can unit test without an API key anywhere near it. Keep the conditionals out of the Jev layer entirely and give the decision method a domain enum to return.

<?php

declare(strict_types=1);

namespace App\Support\Enquiries;

use App\Support\Jev\Answers;

final readonly class EnquiryRouter
{
    public function route(Answers $answers): EnquiryRoute
    {
        // Bulk mail short-circuits everything, whatever it claims to be about.
        if ($answers->noul('is_automated') > 0.8) {
            return EnquiryRoute::Archive;
        }

        return match ($answers->choice('category')) {
            'support' => $this->routeSupport($answers),
            'new_project' => $this->routeLead($answers),
            'recruitment' => EnquiryRoute::Recruitment,
            default => EnquiryRoute::Triage,
        };
    }

    private function routeSupport(Answers $answers): EnquiryRoute
    {
        // Speculative answers, read only now that we know this is support.
        if ($answers->score('severity') >= 1.5 || $answers->noul('mentions_data_loss') > 0.6) {
            return EnquiryRoute::OnCall;
        }

        return EnquiryRoute::SupportQueue;
    }

    private function routeLead(Answers $answers): EnquiryRoute
    {
        $qualified = $answers->score('budget_signal') >= 1.5
            && $answers->noul('named_technology') > 0.5;

        return $qualified ? EnquiryRoute::SalesPriority : EnquiryRoute::SalesBacklog;
    }
}

Low confidence on category is the one case worth gating rather than branching: send it to EnquiryRoute::Triage and let a human look. The thresholds I use for that are the same bands from the ticket triage piece.

Check the cost and latency arithmetic#

The reason this pattern holds up is arithmetic, not vibes. Jev 1.13 charges $0.042 per million input tokens and nothing at all for output tokens. The state is almost always the bulk of the input, so N single-question calls pay for the state N times.

TypeSafe's parallel questions cookbook measures it on a 53,777-character Wikipedia article about the GDPR with 13 questions — 8 Nouls, 2 Choices, 3 Scores:

That's 12.2x cheaper and 10.0x faster, and those are TypeSafe's own published numbers from their cookbook, not measurements I took. Two honest caveats on the speed column: the 2.71s sums thirteen sequential calls, so firing them concurrently narrows the latency gap — but not the 13x token bill. And the saving scales with how much the state dominates the request. A 200-character state with thirteen long questions will not show you 12x.

The cookbook also checks whether batching changes the answers, by asking each question five times under both strategies and comparing run-to-run standard deviation. It doesn't: most answers came back byte-identical across all five repeats either way, and the two questions carrying any sampling noise carried the same amount of it batched and unbatched.

If you want that arithmetic to be visible rather than theoretical, log usage.input_tokens from every response against the enquiry, the same way I log LLM spend in tracking token usage and cost in the Laravel AI SDK. One sum() over a month tells you what the batching is actually saving.

Stop reaching for Http::pool()#

Pooling is the instinct a Laravel developer brings to "I have N API calls to make", and it is exactly wrong here. Pooling fires N concurrent HTTP requests, and each of those requests carries its own copy of the state. You've parallelised the latency and paid the full N-times token cost to do it — the expensive axis was never the round trips.

// Don't. Six requests, six copies of the state, six times the bill.
$responses = Http::pool(fn (Pool $pool) => [
    $pool->as('category')->withToken($key)->post($endpoint, [...]),
    $pool->as('urgency')->withToken($key)->post($endpoint, [...]),
    // ...
]);

// Do. One request, one copy of the state, answers keyed by question id.
$answers = $jev->ask(EnquiryState::for($enquiry), EnquiryQuestions::all());

Pooling earns its keep when you have many different states to evaluate — a thousand enquiries in a nightly backfill, each needing its own request. That's a genuine case for concurrency, and it pairs well with Laravel's Concurrency facade or a queue with a sensible worker count. Mind the published rate limits while you're at it: 80 requests per second and 100K tokens per second on Jev 1.13, and TypeSafe warns those numbers are moving while they scale.

Test the batch with Http::fake()#

Two things are worth asserting, and the second is the one people forget. The first is that your routing logic reads the answers correctly. The second is that the request actually contained every question — because the failure mode of a growing question set is a question that silently stops being sent after a refactor, leaving a branch reading an array key that isn't there.

<?php

declare(strict_types=1);

use App\Support\Enquiries\EnquiryQuestions;
use App\Support\Jev\Jev;
use Illuminate\Support\Facades\Http;

it('sends every question in a single request', function (): void {
    Http::fake([
        'api.typesafe.ai/*' => Http::response([
            'model' => 'jev-1.13.0',
            'answers' => [
                'category' => ['type' => 'choice', 'choice' => 'support', 'probabilities' => ['support' => 0.94], 'confidence' => 0.91],
                'severity' => ['type' => 'score', 'score' => 2.0, 'probabilities' => ['2' => 0.97], 'confidence' => 0.95],
                'mentions_data_loss' => ['type' => 'noul', 'noul' => 0.81],
                'is_automated' => ['type' => 'noul', 'noul' => 0.02],
                // ... the remaining four answers
            ],
            'usage' => ['input_tokens' => 1_204, 'output_tokens' => 96],
        ]),
    ]);

    $answers = app(Jev::class)->ask(['enquiry' => ['body' => 'Our checkout is down.']], EnquiryQuestions::all());

    expect($answers->choice('category'))->toBe('support');

    Http::assertSent(function ($request): bool {
        // One request, and it carries the whole question set.
        return $request->url() === 'https://api.typesafe.ai/v1/systemone'
            && count($request['questions']) === count(EnquiryQuestions::all())
            && array_key_exists('severity', $request['questions']);
    });

    Http::assertSentCount(1);
});

Http::assertSentCount(1) is the regression test for the whole article: the day someone reintroduces a second round trip, that assertion fails. For the routing itself, drive EnquiryRouter directly from a hand-built Answers object across a Pest dataset — it's pure PHP and needs no fake at all. The same split I use for faking agents in the Laravel AI SDK applies here: fake the boundary, test the logic natively.

Gotchas and Edge Cases#

The published speed-up figures disagree with each other. TypeSafe's primitives page currently quotes 11.5x cheaper and 9.6x faster for the same cookbook that reports 12.2x and 10.0x on its own page. They're different runs of the same notebook, not different claims. Quote the cookbook, give the date you read it, and don't put either number in a client proposal as if it were your workload.

Questions in one request cannot see each other's answers. They're evaluated independently against the same state, by design. If a later judgment genuinely needs an earlier answer — because the answer determines what goes into the state, or which options the next Choice offers — that's a real second request. If it merely needs the answer to decide whether to ask, that's speculative fan-out, and it belongs in the first call.

It's 422, not 413. A malformed or oversized question map comes back as 422 Unprocessable Entity with the offending field named in the body, not a 413. The other three documented failures are 401 for a bad key, 429 for rate limiting and 529 when TypeSafe is overloaded. Only the last two are worth retrying, which is why the client above uses retry() with a backoff rather than a blanket retry on every failure.

Mind the shape limits per question type. A Choice takes at most 255 options; a Score needs at least two levels and the API accepts up to ten. Both are easy to breach when you're generating criteria from a database table rather than writing them by hand.

The state is what you need to watch, not the question count. The 32k ceiling covers the state plus your single longest question. If you're near it, summarise or chunk the state — trimming questions to make room is optimising the cheap side of the request.

Wrapping Up#

Build the state once, put every question your code might want into one QuestionSet, send it, and branch in PHP. The marginal cost of the fourteenth question is a few tokens; the marginal cost of the second round trip is the entire state again. If you've got a Http::pool() loop against api.typesafe.ai in your codebase, that's a half-hour refactor with a measurable number attached to it.

From here, confidence-gated ticket triage with Jev covers what to do when the answer comes back uncertain rather than wrong, and verifying LLM output with Jev shows the same batching trick applied to screening what a text model just generated.

FAQ#

Can I ask Jev more than one question in a single request?

Yes, and TypeSafe recommends it. The questions field of POST /v1/systemone is a map of ids to typed question objects, and you can mix Noul, Choice and Score freely in the same request. Every question is evaluated in parallel against the same state, and the answers come back under the same ids you chose.

Does batching questions change Jev's answers?

No. Each question is scored independently against the state, so its answer doesn't depend on what else shares the request. TypeSafe's parallel questions cookbook tested this directly by asking 13 questions five times each under both strategies: most answers were identical on every call either way, and the two questions with any run-to-run variance showed the same variance batched and unbatched.

How is TypeSafe billed — per question or per request?

Per input token, with output tokens free. On Jev 1.13 that's $0.042 per million input tokens. Because the state is sent once per request and usually dominates the token count, the bill is driven by how many times you send the state rather than how many questions you ask about it.

What is speculative fan-out in TypeSafe?

It's the pattern of including questions in a request whose answers only matter on some code paths, then ignoring the ones that turn out to be irrelevant. Asking whether a bug report has reproducible steps costs almost nothing even when the ticket turns out to be a billing query, and it saves a second round trip when it isn't.

How many questions can one TypeSafe request contain?

There's no documented cap on the number of questions. The real limit is the token budget: on Jev 1.13 a request can total 64k tokens across the state and all questions combined, with a further 32k ceiling on the state plus the single longest question. A Choice is capped at 255 options and a Score at ten levels.

Should I use Http::pool() for TypeSafe calls in Laravel?

Not for several questions about the same state — pooling sends the state once per request, so you pay the full token cost and only parallelise the latency. Batch the questions into one call instead. Http::pool() is the right tool when you have many different states to evaluate, such as a backfill over thousands of records, where each really does need its own request.