A Laravel Validation Rule That Scores Spam and Toxicity with Jev

Honeypots and rate limits stop bots. They do nothing about a real human pasting a casino link into your comment form, or an account posting something abusive. LLM moderation is the usual answer, and it's the wrong shape for a form request — you're not waiting three seconds on a text model to tell you whether $request->body is acceptable. TypeSafe's Jev returns a typed score in well under a second, which makes a synchronous ValidationRule practical. Here's the rule, the thresholds, and the fail-open path.

Generate the validation rule#

Scaffold a rule object with Artisan. Laravel 13 generates a class implementing Illuminate\Contracts\Validation\ValidationRule, which has a single validate() method taking the attribute, the value and a $fail closure. There's no passes()/message() split any more, so the rule owns its own message.

php artisan make:rule NotSpam --no-interaction

The rule needs the Jev client, so give it a constructor and resolve it from the container at the call site. This is the same thin Jev wrapper over POST https://api.typesafe.ai/v1/systemone that I built in confidence-gated ticket triage with Jev — there's still no official PHP SDK, so it's about thirty lines of Http::withToken().

<?php

declare(strict_types=1);

namespace App\Rules;

use App\Support\Jev\Jev;
use Closure;
use Illuminate\Contracts\Validation\ValidationRule;

final class NotSpam implements ValidationRule
{
    /** Set when Jev answered, but not confidently enough to reject. */
    private bool $needsModeration = false;

    public function __construct(private readonly Jev $jev) {}

    public function validate(string $attribute, mixed $value, Closure $fail): void
    {
        // Implemented over the next three steps.
    }

    public function needsModeration(): bool
    {
        return $this->needsModeration;
    }
}

Rule objects are not resolved out of the container by the validator, so instantiate it with app() in the FormRequest and keep the reference — you need the same instance afterwards to read the moderation flag.

public function rules(): array
{
    return [
        'body' => ['required', 'string', 'max:5000', $this->notSpam = app(NotSpam::class)],
    ];
}

public function needsModeration(): bool
{
    return $this->notSpam->needsModeration();
}

Ask Jev to score the submission#

A Score question rates the state against ordered, descriptive levels and returns a fractional score, a probabilities map, a legend, and a confidence value. Levels are numbered by their position in criteria, starting at 0, so three levels give you a score between 0.0 and 2.0.

The wording of the levels is the whole game. Jev evaluates each level independently against the state and never sees its number or its neighbours, so "moderately spammy" gives it nothing to match against. The Score docs show numeric-only criteria scoring 0.55 at confidence 0.33 on a report that descriptive levels place at 0.0 with confidence 1.0. Describe situations:

<?php

declare(strict_types=1);

namespace App\Support\Jev;

final class SpamQuestion
{
    /** @var list<string> */
    public const array LEVELS = [
        'Genuine contribution: on-topic, written for other readers',
        'Low-effort or promotional: thin, off-topic, or written to place a link or plug a product',
        'Spam or abusive: unsolicited advertising, scams, harassment, slurs or threats',
    ];

    /** @return array<string, array<string, mixed>> */
    public static function all(): array
    {
        return [
            'spam_level' => [
                'type' => 'score',
                'instructions' => 'How spammy or abusive is this comment?',
                'criteria' => self::LEVELS,
            ],
        ];
    }
}

Keep one Score question to one dimension. Spam and abuse are arguably two things, and if your moderation policy treats them differently, ask two Score questions in the same request — questions are evaluated in parallel, so a second one barely moves the latency. I'd start with the combined scale and split it only when confidence on real submissions turns out to be consistently poor.

Decide using score and confidence#

The score tells you what; the confidence tells you whether to act. Confidence is derived from how the probability is spread across the levels: 1.0 when it all sits on one level, lower the more it spreads. A score of 1.5 at confidence 0.9 is the model saying "clearly promotional, leaning spam". The same 1.5 at confidence 0.2 is the model saying "I can't place this", and rejecting on it means rejecting real people.

So reject on both, and use the flag for everything in between.

public function validate(string $attribute, mixed $value, Closure $fail): void
{
    $answer = $this->score((string) $value);

    if ($answer === null) {
        // Jev was unreachable — see the next step.
        $this->needsModeration = true;

        return;
    }

    $score = (float) $answer['score'];
    $confidence = (float) $answer['confidence'];

    // Top level with the probability actually concentrated there.
    if ($score >= 1.6 && $confidence >= 0.8) {
        $fail('validation.not_spam')->translate();

        return;
    }

    // Promotional, or spammy-but-uncertain. Accept it, hide it, let a human look.
    $this->needsModeration = $score >= 1.0 || $confidence < 0.5;
}

Those two numbers are a starting point, not a configuration default — test them against your own moderated backlog before you trust them. The sensible instinct is to tune them per action: rejecting a comment outright deserves a higher bar than queueing it for review, exactly as gating actions on confidence bands does for ticket routing.

Store the verdict on the model so the flag survives the request:

$comment = Comment::create([
    ...$request->validated(),
    'needs_moderation' => $request->needsModeration(),
]);

Fail open when Jev is unavailable#

A third-party API inside synchronous validation is a liability unless you decide up front what happens when it's slow or down. Blocking a legitimate comment because TypeSafe returned a 529 is strictly worse than publishing a borderline one and reviewing it later. So: short timeout, no retries in the request path, rescue() to swallow the failure, moderation flag on the way out.

The timeout() call below is a one-line wither on the client (return new self($this->apiKey, $this->model, $seconds);), because the triage client's default of 15 seconds is right for a queued job and far too generous for a form post.

use Illuminate\Support\Facades\Cache;

/** @return array<string, mixed>|null */
private function score(string $body): ?array
{
    if (trim($body) === '') {
        return null;
    }

    // Same body, same verdict — a failed resubmit shouldn't pay twice.
    $key = 'spam-score:'.hash('xxh128', $body);

    return Cache::remember($key, now()->addDay(), fn (): ?array => rescue(
        fn (): ?array => $this->jev
            ->timeout(1.5)   // Hard ceiling on added request latency.
            ->ask($body, SpamQuestion::all())['spam_level'] ?? null,
        null,                // Fail open.
        report: true,        // But still tell Sentry.
    ));
}

Two things worth being explicit about. rescue() with report: true means outages surface in your error tracker instead of silently degrading moderation quality for a week. And caching a null verdict for a day is wrong — if you want the cache to hold only real answers, check for null before writing rather than leaning on Cache::remember. A 1.5 second timeout is also a real tax on a form post; if your form is doing live field validation through Laravel Precognition, exclude this rule from precognitive runs with $request->isPrecognitive() so you're not scoring every keystroke.

Test the rule with Http::fake#

The decision logic is ordinary PHP, so test it as such — no API key, no network. Fake the endpoint with a response shaped like a real Score answer, and cover the three bands plus the outage. A dataset keeps the boundary cases honest, which is where this rule will actually break.

<?php

declare(strict_types=1);

use App\Rules\NotSpam;
use Illuminate\Http\Client\ConnectionException;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Validator;

function scoreResponse(float $score, float $confidence, array $probabilities): array
{
    return [
        'model' => 'jev-1.13.0',
        'answers' => [
            'spam_level' => [
                'type' => 'score',
                'score' => $score,
                'confidence' => $confidence,
                'probabilities' => $probabilities,
            ],
        ],
        'usage' => ['input_tokens' => 148, 'output_tokens' => 18],
    ];
}

function validateBody(): array
{
    $rule = app(NotSpam::class);
    $validator = Validator::make(['body' => 'whatever'], ['body' => [$rule]]);

    return [$validator->passes(), $rule->needsModeration()];
}

it('rejects confident spam', function (): void {
    Http::fake(['api.typesafe.ai/*' => Http::response(
        scoreResponse(2.0, 1.0, ['0' => 0.0, '1' => 0.0, '2' => 1.0])
    )]);

    expect(validateBody())->toBe([false, false]);
});

it('flags spam the model is unsure about', function (): void {
    // Probability split across opposite ends: score 1.0, confidence 0.0.
    Http::fake(['api.typesafe.ai/*' => Http::response(
        scoreResponse(1.0, 0.0, ['0' => 0.5, '1' => 0.0, '2' => 0.5])
    )]);

    expect(validateBody())->toBe([true, true]);
});

it('passes a genuine comment untouched', function (): void {
    Http::fake(['api.typesafe.ai/*' => Http::response(
        scoreResponse(0.0, 1.0, ['0' => 1.0, '1' => 0.0, '2' => 0.0])
    )]);

    expect(validateBody())->toBe([true, false]);
});

it('fails open when TypeSafe is unreachable', function (): void {
    Http::fake(fn () => throw new ConnectionException('cURL error 28: timed out'));

    expect(validateBody())->toBe([true, true]);
});

That last test is the one that matters. It's also the one that silently stops testing anything if someone later adds ->retry() to the client, so assert on the request count if your client grows retry behaviour. The same Http::fake() approach covers the wider Jev integration — see verifying LLM output with Jev for the guardrail variant.

Gotchas and Edge Cases#

A score is not a distribution. 1.0 can mean all the probability sits on level 1, or that it's split evenly between levels 0 and 2. Those are completely different submissions. If your policy cares about the difference, read probabilities directly rather than thresholding the score alone.

Confidence 1.0 is not correctness. It only means the returned distribution landed on one level. The model can be confidently wrong, which is why the reject branch needs a human-reviewable audit trail — store the score, confidence and the jev-1.13.0 model string alongside the comment.

Validation runs before authorization in some flows. If your rule is expensive and your route is public, you've built a paid endpoint for anyone with curl. Put rate limiting on the route before you put AI in the validator.

Array values will surprise you. validate() receives mixed. A client posting body[]=... hands you an array, (string) $value throws, and you've turned a spam check into a 500. Cast defensively or chain string first in the rule array.

Don't batch-score in a loop. If you're moderating a backlog, send the submissions as separate questions in one request rather than one request per row — same latency, a fraction of the input tokens.

Wrapping Up#

Start with the three levels above, log every score and confidence for a fortnight without acting on them, then set your thresholds from your own data rather than mine. The reject-plus-flag split is the part that generalises: it's the same shape as the confidence bands in Jev ticket triage, and if you're already moderating LLM output you can reuse the questions from Laravel AI SDK agent guardrails rather than writing a second rubric.

FAQ#

How do I detect spam in Laravel forms?

Layer it. Honeypot fields and rate limiting remove automated submissions cheaply, and a content check handles the humans those miss. For the content check, a custom ValidationRule that sends the submission to a classifier and thresholds the result keeps the decision inside normal validation, so the error surfaces on the field like any other rule failure.

How do I create a custom validation rule in Laravel?

Run php artisan make:rule NotSpam. Laravel 13 generates a class implementing Illuminate\Contracts\Validation\ValidationRule with one method, validate(string $attribute, mixed $value, Closure $fail). Call $fail() with a message or a translation key when the value is invalid, and return without calling it when it's fine. Add the rule instance to the rules array for that field.

Can AI moderate comments in real time?

With a System One model, yes. Jev returns a typed score in well under a second, which is inside the budget for a form post, where a chat model taking two or three seconds is not. The practical constraints are a short timeout and a defined fail-open path, so an outage degrades to human review rather than blocking submissions.

What happens if the AI service is down during validation?

That is your decision, and you should make it deliberately. Failing closed rejects legitimate comments during someone else's outage, so the usual choice is to fail open: wrap the call in rescue(), accept the submission, and set a moderation flag so a human sees it. Report the exception as well, otherwise moderation quietly stops working and nothing tells you.