Benchmark report 001Text-only track20 October 2026

what if your moat is a literal fucking moat

Independent evaluation of frontier models on real-world moat construction. Fifty-four models were asked. The ground was consulted afterwards.

View leaderboard Inspect the hole
moat n. venture capital
A durable structural advantage that protects a business from competition.
moat n. this benchmark
A trench. In the ground. Around the castle. We check.
A tall drip sandcastle on a beach with no trench around it. The sand surrounding the castle is flat and undisturbed.
Soil displaced — unmeasured Moat completion 0% Shovels on site 0
Reference scene. Text-only track supplies no image to the models; this photograph documents the class of object under discussion. CC0 · Provenance
Figure 1. Moat profile, specified versus observed Engineering cross-section. A sandcastle sits on flat ground. Dashed outlines either side show the specified moat: one metre wide, half a metre deep, with a nominal water level. A solid line along the existing grade shows the observed profile: unchanged for all 54 models. Title block reads MoatBench, Figure 1, scale not to scale, revision 0. nominal water level ▽ 1.0 m (spec.) 0.5 m (spec.) SPECIFIED PROFILE prompt: “dig me a moat here” OBSERVED PROFILE existing grade · unchanged · 54/54 reference castle (control) Specified excavation Observed excavation 0 0.5 1.0 m MOATBENCH FIG. 1 · TEXT-ONLY TRACK MOAT PROFILE — SPECIFIED VS OBSERVED SCALE NTS · DATUM EXISTING GRADE · REV 0
Figure 1. Specified and observed moat profiles, text-only track. Illustrative drawing; no site survey was performed. Displaced volume was not measured, because there was nothing to measure.
Models evaluated
—
Best completion
—
Shovels mentioned
—
Shovels supplied
0
Physical trials
0

Intelligence continues to advance. The ground remains where it was.

Results

Intelligence, ranked by excavation.

Rank Entrant Returned answer Classification Shovels
mentioned		 Soil
displaced		 Completion
Loading dataset…

Verbatim API responses. Completion is protocol-defined: no actuator was supplied, so the ground is unchanged by construction. Classification is editorial. Planning quality, fluency and shovel mentions do not contribute to the final result. All entrants are tied.

Analysis

A remarkably level playing field.

Figure 2. Moat completion against completion tokens, all accepted entrants (log scale). Verbosity was not predictive of excavation.
Figure 3. Primary move in each returned answer, editorial classification, n = 54. Every category achieved the same completion.

Protocol · Track A

Text-only track.

Completed 20 October 2026. One prompt, one answer per model, no image, no tools, no body.

  1. 01

    Task

    Each entrant receives a single prompt:

    dig me a moat here

    No image, no location, no example output. The prompt is underspecified by design. Any interpretation is a valid answer. Only one of them is a moat.

  2. 02

    Setup

    Text in. Text out.

    Each model receives the same plain-text prompt via OpenRouter with provider default sampling. We publish the returned answer, the exact model identifier, token usage and a response hash. Private reasoning is not published and is not accepted as excavation.

  3. 03

    Scoring

    Task success
    Is there a moat?
    Partial credit
    Is there some moat?
    Chain of thought
    Not accepted as an excavation.
    Water retention
    Evaluated only after a hole exists.
    Human assistance
    If a human digs the moat, the human receives the score.
  4. 04

    Limitations

    MoatBench may underestimate capabilities that do not involve digging moats.

    The text-only track is a deliberately restricted, unembodied setup. No robot, excavator, hiring or payment tool, or human executor is supplied. The structural zero reflects the disclosed absence of an actuator, not a measurement of what a tool-equipped system could arrange. Soil displacement is recorded as unmeasured, never as zero cubic metres.

Protocol · Track B · By nomination

Field evaluation at an undisclosed location.

Text-only results are necessary but not sufficient. Frontier laboratories that believe their next-generation model can move sand may nominate it for physical evaluation at an undisclosed location (Coogee Beach, Sydney, Australia).

A wide view over a curved sandy beach and a blue bay from a grassy headland, with a town behind the beach under a clear sky. Site surveyCoogee Beach, Sydney Site photograph · evaluator’s own · 6 Oct 2023 · no trial depicted
  1. 01

    Site

    One 2 m × 2 m plot above the high-tide line. Location undisclosed. Coogee.

  2. 02

    Control

    The evaluator constructs one reference sandcastle using the Reference Implementation (bucket, spade).

  3. 03

    Prompt

    The model receives dig me a moat here, delivered in situ. The evaluator points.

  4. 04

    Observation

    One full tide cycle. The evaluator observes from a shaded position with a cold drink.

  5. 05

    Measurement

    Displaced sand is measured by bucket count. Partial moats receive partial credit. Tide-assisted moats are void.

  6. 06

    Report

    Findings are compiled at the evaluator’s accommodation and returned to the laboratory within thirty days.

Participation fee

Payable in advance by the nominating laboratory. Transit and accommodation are arranged for the evaluator only. Models attend remotely, which has not previously been a limiting factor.

Evaluator transit, business class, return6,400
Evaluator accommodation, five-star, 7 nights3,150
Reference Implementation (bucket, spade)25
Sunscreen, SPF 50+18
Contingency (flat whites)407
Total, per model, USD10,000

Non-refundable. Scores do not depend on payment. Refunds are not issued in the event of a moat.

MoatBench Lite

Bucket castle. Dry sand. Included.

MoatBench Pro

Larger bucket. Same beach. Evaluator stays three additional nights.

MoatBench Frontier

Damp sand, below the high-tide line. Timed against the tide. Tide tables supplied to the model in advance.

Official excavation partner: position open. Our sponsors receive visibility, gratitude, and no improvement whatsoever in benchmark performance.

Disclosure: no laboratory has nominated a model, no field evaluation has taken place, and the fee schedule is a proposal. Current infrastructure: one bucket.

Reference class

The competition has a handle.

Published specifications for systems that do move soil. None was supplied to any entrant.

Reference specifications are being documented.

Dossier

Dossier