A client had 4,000 scanned membership forms in S3 and one question about each: where is the signature block? The usual answers are a hosted vision API with a fixed label set that has never heard of "signature", training a detector on data nobody has labelled, or asking a chat model for coordinates it cannot be held to. A Jev box question in Laravel does it in an afternoon — with caveats I will not bury at the bottom.
Understand what a Box question returns#
A Box is a System One question type that locates one thing in an image. It takes a type of "box" and instructions describing the target and which image to look in — "the signature block in form" is enough. It takes no criteria, unlike Choice and Score. The answer is the smallest rectangle containing the target, as four edges measured from the top-left corner.
{
"model": "jev-1.14.0",
"answers": {
"signature": {
"type": "box",
"box": { "top": 0.81, "bottom": 0.93, "left": 0.52, "right": 0.88 },
"confidence": 0.86
}
},
"usage": { "input_tokens": 290, "output_tokens": 21 }
}
left and right are fractions of the width, top and bottom fractions of the height. That is the detail that makes this pleasant in a Laravel app. Most apps store several derivatives of every upload — a 320px thumbnail, a 1200px web copy, the original. Pixel coordinates are only valid for the exact image you sent. The same four fractions crop all three.
Two constraints before you build on it. Box needs Jev 1.14 — images work on 1.13, Box does not. And both images and Box are what TypeSafe calls dreamer-ready: the request format, the answers and the limits can change between releases. Build prototypes on it, pin the model string rather than jev-latest, and keep the integration behind an interface you can swap.
Turn images on with the danger zone header#
Images and Box questions are off by default. Every request that carries an image or a Box question must send TypeSafe-Danger-Zone-Images: true or the API returns a 400 — not a helpful validation error about your payload, just a refusal. Put it in the client, not at the call site, so nobody forgets it.
<?php
declare(strict_types=1);
namespace App\Support\Jev;
use Illuminate\Http\Client\PendingRequest;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Str;
final readonly class JevImages
{
public function __construct(
private string $apiKey,
// Pin the exact version. Box behaviour changes between releases.
private string $model = 'jev-1.14.0',
) {}
private function request(): PendingRequest
{
return Http::baseUrl('https://api.typesafe.ai/v1')
->withToken($this->apiKey)
->withHeaders(['TypeSafe-Danger-Zone-Images' => 'true'])
->timeout(30);
}
}
Prepare the upload before you send it#
The API rejects images outside its limits rather than resizing them to fit, and one bad image fails the whole request — there are no partial answers. Each side must be between 280 and 1568 pixels, total pixels between 78,400 and 2,458,624, file size at most 16 MiB, PNG or JPEG only, still images only. A 12-megapixel phone photo is over the ceiling on every count.
The subtle one is orientation. The API strips all metadata and does not apply EXIF orientation, so a photo that looks upright in Preview can reach the model on its side — and a box computed against a sideways image lands nowhere near the signature. Intervention Image v3 auto-orients on read by default, which fixes this for free as long as you send the re-encoded bytes rather than the original file.
<?php
declare(strict_types=1);
namespace App\Support\Jev;
use Intervention\Image\ImageManager;
final readonly class PreparedImage
{
private function __construct(
public string $contents,
public int $width,
public int $height,
) {}
public static function fromContents(string $raw): self
{
// v3 applies EXIF orientation on read unless autoOrientation is disabled.
$image = ImageManager::gd()->read($raw);
// Longest side to 1568; scaleDown never upscales a small image.
$image->scaleDown(1568, 1568);
return new self(
// Baseline JPEG is the safest thing to hand the API.
contents: (string) $image->toJpeg(quality: 90),
width: $image->width(),
height: $image->height(),
);
}
}
Keep the width and height you actually sent. You need them later, and they are not the dimensions of the file in storage.
Send the image as a multipart request#
Multipart is the better transport: base64 inflates each image by about a third and that growth counts against the 32 MiB request body limit. The JSON body goes first, as a part literally named request with content type application/json. Each image is its own part, and the asset's part field names it. Laravel's Http::attach() builds exactly this, and attaches in the order you call it.
/**
* @param array<string, array<string, mixed>> $questions
* @return array<string, mixed>
*/
public function ask(PreparedImage $image, array $questions): array
{
// A fresh UUID per request: references are matched as plain text
// anywhere in the payload, so short keys can collide with your prose.
$reference = (string) Str::uuid();
$payload = [
'model' => $this->model,
'state' => ['form' => $reference],
'assets' => [$reference => ['type' => 'image', 'part' => 'upload']],
'questions' => $questions,
];
return $this->request()
->attach('request', json_encode($payload, JSON_THROW_ON_ERROR), headers: ['Content-Type' => 'application/json'])
->attach('upload', $image->contents, 'upload.jpg', ['Content-Type' => 'image/jpeg'])
->post('/systemone')
->throw()
->json('answers');
}
Three multipart rules bite in practice. Part names must be unique and non-empty, request is reserved for the body, and every asset must be referenced somewhere in state, instructions or criteria — an orphan asset is a 400. An image part whose content type is anything other than image/png, image/jpeg, image/jpg or application/octet-stream is also a 400.
Convert the fractions to pixels with a value object#
The conversion is four multiplications, which is exactly why it belongs in one tested class rather than scattered across jobs and controllers. Model the answer as a readonly value object that carries the fractions and knows how to project them onto any size, plus a pad() for the cases below.
<?php
declare(strict_types=1);
namespace App\Support\Jev;
final readonly class BoundingBox
{
public function __construct(
public float $top,
public float $bottom,
public float $left,
public float $right,
public ?float $confidence,
) {}
/** @param array<string, mixed> $answer A single `box` answer from the API. */
public static function fromAnswer(array $answer): self
{
$box = $answer['box'];
return new self(
top: (float) $box['top'],
bottom: (float) $box['bottom'],
left: (float) $box['left'],
right: (float) $box['right'],
// Confidence is nullable: the model does not always provide it.
confidence: isset($answer['confidence']) ? (float) $answer['confidence'] : null,
);
}
/** Grow the box by a fraction on every side, clamped to the image. */
public function pad(float $fraction): self
{
return new self(
top: max(0.0, $this->top - $fraction),
bottom: min(1.0, $this->bottom + $fraction),
left: max(0.0, $this->left - $fraction),
right: min(1.0, $this->right + $fraction),
confidence: $this->confidence,
);
}
/**
* The fractions, ready to persist on the model so a later re-crop
* needs no second request.
*
* @return array{top: float, bottom: float, left: float, right: float, confidence: float|null}
*/
public function toArray(): array
{
return [
'top' => $this->top,
'bottom' => $this->bottom,
'left' => $this->left,
'right' => $this->right,
'confidence' => $this->confidence,
];
}
/**
* @return array{x: int, y: int, width: int, height: int}
*/
public function toPixels(int $imageWidth, int $imageHeight): array
{
$x = (int) round($this->left * $imageWidth);
$y = (int) round($this->top * $imageHeight);
return [
'x' => $x,
'y' => $y,
// Round the far edge, then subtract, so the crop never overruns.
'width' => max(1, (int) round($this->right * $imageWidth) - $x),
'height' => max(1, (int) round($this->bottom * $imageHeight) - $y),
];
}
}
Note the ?float $confidence. The OpenAPI schema declares it nullable and the API's own Box example returns "confidence": null. Code that does $answer['confidence'] < 0.6 on a null gets true in PHP 8 and silently takes the low-confidence branch for every answer. Treat null as "unknown", not as zero.
Pair the Box with a Noul before you crop#
A Box answer is always a box. There is no "not found" — ask where the signature is on an unsigned form and you still get four edges, pointing at whatever looked most signature-ish. This is the single biggest way to ship a broken feature on top of Box.
The fix is one extra question in the same request. Ask a Noul whether the target is in the image at all, and ignore the box when the Noul answer is low. Since Jev charges for the state once per request, the second question costs a few tokens — the same economics behind batching thirteen questions into one Jev call.
$answers = $jev->ask($prepared, [
'is_signed' => [
'type' => 'noul',
'instructions' => 'Is there a handwritten signature anywhere in `form`?',
],
'signature' => [
'type' => 'box',
'instructions' => 'the handwritten signature in `form`',
],
'document_type' => [
'type' => 'choice',
'instructions' => 'What kind of document is `form`?',
'criteria' => [
'membership' => 'A membership application form.',
'mandate' => 'A direct debit mandate.',
'other' => 'Anything else.',
],
],
]);
One round trip, one copy of the image, three answers. That is the pattern I reach for every time: the Box tells you where, the Noul tells you whether to believe it, and a Choice gives you the routing key for free — the same confidence-gated triage idea applied to pixels.
Gate the crop on confidence and pad when unsure#
TypeSafe publishes no recommended threshold for Box, and says plainly to tune it on your own images. So pick a constant, write down why, and measure it against a set of forms you have checked by hand. When the answer is below the line, the choice is not crop-or-don't: pad the box outward first, because a crop that cuts the signature in half is worse than a loose one, and both are worse than showing the whole page.
final class SignatureRegion
{
// Tuned on 120 hand-checked membership forms, not a TypeSafe recommendation.
private const MIN_CONFIDENCE = 0.6;
private const PAD = 0.05;
public static function resolve(array $answers): ?BoundingBox
{
// Box always returns edges, so the Noul decides whether a crop happens.
if ((float) $answers['is_signed']['noul'] < 0.7) {
return null;
}
$box = BoundingBox::fromAnswer($answers['signature']);
// Null confidence means unknown, so treat it like a low score.
$confidence = $box->confidence ?? 0.0;
return $confidence < self::MIN_CONFIDENCE
? $box->pad(self::PAD)
: $box;
}
}
A worked example of why this matters. On a landscape scan where the box came back at 0.41 confidence, the edges enclosed the printed "Signed:" label and about a third of the actual signature. Cropped tight, the user sees a thumbnail of the word "Signed" and concludes the system is broken. Padded by 5% on each side, the same answer produces a usable crop. Below the Noul gate, they see the whole page and nothing looks broken at all.
Crop in a queued job and store the fractions#
Keep all of this off the request cycle. The call is a network round trip plus image encoding on both sides, so it belongs in a queued job the same way background agent runs do. Store the fractions on the model alongside the derived crop — when the model improves or your padding changes, you re-crop from the stored fractions without paying for the request again.
<?php
declare(strict_types=1);
namespace App\Jobs;
use App\Models\ScannedForm;
use App\Support\Jev\BoundingBox;
use App\Support\Jev\JevImages;
use App\Support\Jev\PreparedImage;
use App\Support\Jev\SignatureRegion;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queueable;
use Illuminate\Support\Facades\Storage;
use Intervention\Image\ImageManager;
final class ExtractSignatureRegion implements ShouldQueue
{
use Queueable;
public int $tries = 3;
public function __construct(private readonly ScannedForm $form) {}
public function handle(JevImages $jev): void
{
$original = Storage::disk('scans')->get($this->form->path);
$prepared = PreparedImage::fromContents($original);
$box = SignatureRegion::resolve($jev->ask($prepared, [
'is_signed' => ['type' => 'noul', 'instructions' => 'Is there a handwritten signature anywhere in `form`?'],
'signature' => ['type' => 'box', 'instructions' => 'the handwritten signature in `form`'],
]));
if ($box === null) {
$this->form->update(['signature_box' => null, 'signed' => false]);
return;
}
// Crop the ORIGINAL, not the downscaled copy we sent. Same fractions, better pixels.
$image = ImageManager::gd()->read($original);
$pixels = $box->toPixels($image->width(), $image->height());
$image->crop($pixels['width'], $pixels['height'], $pixels['x'], $pixels['y']);
Storage::disk('scans')->put(
$path = "signatures/{$this->form->id}.jpg",
(string) $image->toJpeg(quality: 90)
);
$this->form->update([
'signature_box' => $box->toArray(),
'signature_path' => $path,
'signed' => true,
]);
}
}
That last point is the whole payoff of fractional edges: you ask about a 1568px copy because that is all the API accepts, and you crop the 4000px original because that is what the user wants to look at. If your disks and derivatives are not set up for that, Laravel's file storage layer handles it cleanly.
Test the pixel maths with a faked response#
The API call is not what breaks. The conversion is — off-by-one on the far edge, transposed width and height, a null confidence taking the wrong branch. Record one real Box response, fake it, and assert the pixels against a known image size.
<?php
declare(strict_types=1);
use App\Support\Jev\BoundingBox;
it('converts fractional edges to a pixel crop', function (): void {
$box = BoundingBox::fromAnswer([
'type' => 'box',
'box' => ['top' => 0.25, 'bottom' => 0.75, 'left' => 0.10, 'right' => 0.60],
'confidence' => 0.92,
]);
expect($box->toPixels(1000, 800))->toBe([
'x' => 100,
'y' => 200,
'width' => 500,
'height' => 400,
]);
});
it('treats a null confidence as unknown rather than zero', function (): void {
$box = BoundingBox::fromAnswer([
'type' => 'box',
'box' => ['top' => 0.1, 'bottom' => 0.2, 'left' => 0.1, 'right' => 0.2],
'confidence' => null,
]);
expect($box->confidence)->toBeNull();
});
Weigh Box against a trained detector#
Box is not a replacement for a purpose-trained object detector, and pretending otherwise gets people burned. A YOLO-class model on fixed classes is faster, cheaper per image at volume, deterministic, and runs on your own hardware. If you are processing a million delivery photos a day looking for the same three things, train the detector.
Box wins on the other axis. The target is open-vocabulary — "the signature", "the damaged corner of the box", "the price sticker" — and needs no labelled data, no training run and no deployment. It is one HTTP call. For a prototype due Friday, a long tail of one-off targets, or a volume measured in thousands rather than millions, that trade is obviously right. Just know which side of it you are on before the volume grows.
Gotchas and Edge Cases#
One box per question. Box finds one target. If the page has three signatures, you get one box and no indication there were others. Describe the one you want so only it fits — "the signature nearest the bottom of form" — or add a Choice with options none, one and several, and split the image into regions when the answer is several.
Small targets in large images get missed. A barcode a few dozen pixels wide on a 1568px scan is below what the model reliably sees, and downscaling to meet the limits makes it worse. Find the region first with a Choice whose options are tiles of the image, ask the Box on that tile, then convert the tile's edges back to fractions of the full image.
Every reference use re-sends the image. References are matched as plain text wherever they appear, and each occurrence adds another copy of the image to the expanded request, against a 32 MiB ceiling. Put an image several questions need in state once and refer to it by path, rather than pasting the reference into each question's instructions.
Short references collide. A reference of cat also matches inside the word category in your state. Use a fresh UUID per image per request.
Intervention v4 renamed the entry points. The code here is v3 (ImageManager::gd()->read()). On v4 that is ImageManager::usingDriver(Driver::class)->decodePath(), and encoding moves to encodeUsingFormat(). Check which major you are on before copying.
Progressive JPEGs and HEIC uploads. The API wants baseline JPEG or PNG. Phone uploads frequently arrive as HEIC and progressive JPEG; re-encoding through Intervention as shown normalises both, which is a second reason not to forward the original bytes.
Wrapping Up#
Build it in this order: Noul first to decide whether the thing exists, Box second to find it, confidence gate third, crop last. Store the fractions, not just the cropped file, and pin jev-1.14.0 rather than jev-latest so a release does not quietly change your crops. Keep it behind an interface while the feature stays dreamer-ready.
If you are putting Jev in front of user-generated images, the same confidence discipline applies to text — see the Jev validation rule that scores spam and toxicity. And if you need description rather than location, a vision LLM is the better tool: analysing images with the Laravel AI SDK covers that side.
FAQ#
What is a Box question in TypeSafe Jev?
A Box is a System One question type that locates one thing in an image. You give it a type of "box" and instructions describing the target, such as "the coffee mug in photo", and the answer is the smallest rectangle containing it. The rectangle comes back as four edges — top, bottom, left and right — each a fraction of the image's width or height from the top-left corner, plus a confidence value that may be null.
How do I find the position of an object in an image with Jev?
Send the image as an asset in a System One request, reference it from the state, and ask a Box question whose instructions name both the target and the image. You need the TypeSafe-Danger-Zone-Images: true header and a Jev 1.14 model, because Box does not run on 1.13. Multiply the returned fractions by your image's pixel dimensions to get a crop rectangle.
Are Jev image features production ready?
No. TypeSafe labels images and Box questions dreamer-ready, which means the request format, the answers and the documented limits can change between releases, and accuracy will move as the models are updated. They are good for prototypes and internal tooling. Anything customer-facing should sit behind a review step, a pinned model version and an interface you can replace.
How do I send an image to the TypeSafe API from Laravel?
Use a multipart request built with the HTTP client's attach() method. Attach the JSON body first as a part named request with content type application/json, then attach the image as its own part and set that part's name in the asset's part field. Multipart is preferred over base64 in JSON because base64 inflates the payload by roughly a third, and that extra size counts against the 32 MiB request body limit.
How do I convert Jev box fractions to pixels?
Multiply left and right by the image width and top and bottom by the image height, then round. The x and y offsets are the rounded left and top; the width and height are the rounded far edges minus those offsets, which avoids an off-by-one overrun at the boundary. Because the edges are fractions rather than pixels, the same answer works against a thumbnail, a web-sized copy and the full-resolution original.
Can I ask multiple questions about the same image in one request?
Yes, and you should. Every question in a request is evaluated against the same state, so putting the image in state once and asking a Box, a Noul and a Choice together costs barely more than asking one of them. This is the recommended way to work around Box always returning a box: the Noul tells you whether the target is present before you act on the coordinates.
How does Jev Box compare to a traditional object detection model?
A trained detector wins on throughput, cost per image, latency and determinism when the classes are fixed and the volume is high. Box wins when the target is described in words rather than drawn from a label set, when there is no training data, and when the deadline is this week. For thousands of images with varied targets, Box is the pragmatic choice; for millions with three fixed classes, train the detector.