startthat.work home

CK-14 Data leakage audit (train-test contamination check; model information sheet)

Confirm that no information from the test items or the outcome entered the model, prompt or features, and produce a leakage checklist and a model information sheet.

Check extracted resultsEvidence: DocumentedVerification technique, runs in step 6Strongest statement: describe

What it does

Confirms that no information from the test items or the outcome entered the model, prompt or features. Artefact: a leakage checklist and a model information sheet. Read which of the eight leakage types applies.

When to use it

Use when a predictive or classification claim is about to be reported and you need to know whether test information leaked into development.

When it does not fit

Fits any predictive or classification claim. Misfit: prompt examples drawn from the evaluation set. Pre-training exposure is a further route that was not confirmed.

Result format

The result is a leakage checklist with one line per leakage type, marking whether it applies, together with a model information sheet. It names the leakage types that apply and says what information entered the model, prompt or features.

Requirements on the material

PropertyRequirement typeRequirement
Content typeAlways appliesrequires=text; requires=tabular
Unit of analysisAlways appliesrequires=dataset
Level of detail availableAlways appliesrequires=record-level
Appraises source credibilityApplies in some casesno

Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materiala documented split, the modelling code or prompt, and the feature list
Additional sources needednone
Not for (rules)none
Not for (text)Wrong choice when prompt examples were drawn from the evaluation set and cannot be replaced. Exposure during model pre-training is a further way that this audit cannot report.

Choosing and ordering

Kind of techniqueVerification technique, runs in step 6
Phase tagvalidate
Other phase tagsanalyse
Kind of findingrisk
Question answeredhow much
Strongest statement it supportsdescribe
Needs as input
Makes as output
Must exist firstNone
Loop targetreturn to analyse

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.

StakesCheckPerson involvedCounts as a different method
exploratoryspot checkoptionalno
decision-informingspot checkoptionalno
decision-informingleakage checknoneno
decision-informingprovenance lognoneno
high-stakesspot checkoptionalno
high-stakesleakage checknoneno
high-stakesprovenance lognoneno
high-stakesindependent reviewneededno
high-stakesexpert reviewneededno

Failure modes and limits

Model suitabilityhuman judgement required
Typical model failureCannot report what it saw in pre-training.
Error sourcesinput data; data transformation; coder or rater judgement
Certainty limitsCertainty drops when the development steps are not documented (risk of bias) and when pre-training exposure cannot be ruled out (indirectness).

Evidence and tag basis

Documented. Leakage errors affected 329 papers across 17 fields, and none could be detected by reading the papers alone (abstract, 2022). (Kapoor and Narayanan 2022, arXiv 2207.07048)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.