Built backwards from outcomes: ground truth that is a matter of record

Our corpus is built from the complete documentary records of real financial events: the internal financial exhibits, credit-agreement terms, and deal documents produced in the course of a transaction, together with the public filings and reporting that surrounded it. Because each record is complete and its outcome known, tasks can be constructed backwards, from answers to questions: what a good answer looks like is a matter of record rather than of an author's judgment, so grading criteria are fixed before any model is run. A forward-built environment — questions first, answers second — must either accept a judge's opinion as ground truth or confine itself to trivially checkable answers; construction from outcomes removes that choice.

What a good answer looks like is a matter of record rather than of an author's judgment.

—Provenance

Every source is tracked through a ledger recording how it was retrieved and whether it survived a two-stage audit of its content. The raw retrieved text is kept alongside the cleaned form, and any human intervention in curation is recorded, so the full chain — task to claim to underlying document — is open to inspection. Every factual claim used in task construction carries a verbatim quotation from its source and, where determinable, the date at which the fact became knowable.

The recorded knowability dates are what make point-in-time evaluation enforceable rather than aspirational. Temporal boundaries are enforced in two layers: it is built into the task, in that the supplied materials stop at the anchor date, which is reiterated in the question prompt, and it is enforced again in grading, where an answer that reasons from hindsight suffers a critical loss in reward.

—Memorisation

Our internal exhibits appear in no public corpus, so performance on tasks built from them cannot be explained by memorisation, which only adds to the fact that the evaluation discriminates on reasoning rather than recall — and it stays that way as training sets grow.

—The task roster

A single documented deal, processed this way, yields tasks in the low hundreds — arranged over a grid of anchor dates and seven reasoning categories, from diagnostic to counterfactual. Our reference case carries 117 authored tasks across four anchor dates; empty cells in that grid reflect what each anchor's information set can support, not gaps in coverage. The corpus behind it currently supports production of more than 20,000 graded tasks, and it is not a fixed inventory: documented deals keep entering the pipeline, and each brings its own task roster.

An anonymisation layer allows the corpus to be used for training without exposing the judgments embedded within it.

—Access

We publish the framework — how provenance is tracked, how anchors are enforced, what the grading rewards — and the results it produces (see Where equal-looking models come apart). The task text, the rubrics, and the production pipeline are not published: unsealed evaluation material stops measuring anything the day it leaks. Access is under evaluation agreement: tech@dissei.credit.