startthat.work home

CK-26 Independent human review of model output (expert review; adjudication; human oversight)

Specify who reviews which outputs, with what rubric and with authority to overrule, and produce a review log with decisions.

Check extracted resultsEvidence: DocumentedVerification technique, runs in step 6Strongest statement: describe

What it does

Specifies who reviews which outputs, with what rubric and with authority to overrule. Artefact: a review log with decisions. Read the overruled items and the reasons.

When to use it

Use when an output will inform a decision and a person who did not build it must check and be able to overrule it.

When it does not fit

Fits any output used in a decision. Misfit: reviewers see only items the model approved (automation bias, inference). Neither source gives sample sizes or reviewer numbers.

Result format

The result is a review log with one row per output reviewed, giving the reviewer, the rubric applied, the decision and the reason. It states who reviews which outputs and lists the overruled items with the reasons.

Requirements on the material

PropertyRequirement typeRequirement
Unit of analysisAlways appliesrequires=claim; requires=document; requires=record
Appraises source credibilityApplies in some casesno

Not restricted on: Structure, Content type, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialat least one reviewer who did not develop the model, and a written rubric
Additional sources neededreference structure
Not for (rules)none
Not for (text)Wrong choice when reviewers see only the items the model approved, which invites automation bias. Neither source gives sample sizes or reviewer numbers.

Choosing and ordering

Kind of techniqueVerification technique, runs in step 6
Phase tagvalidate
Other phase tagsconclude
Kind of findingrisk
Question answeredwhat next
Strongest statement it supportsdescribe
Needs as input
Makes as output
Must exist firstNone
Loop targetreturn to analyse

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.

StakesCheckPerson involvedCounts as a different method
exploratoryspot checkoptionalno
decision-informingspot checkoptionalno
decision-informinghuman review sampleneeded; left out below high stakesno
decision-informingprovenance lognoneno
high-stakesspot checkoptionalno
high-stakeshuman review sampleneededno
high-stakesprovenance lognoneno
high-stakesindependent reviewneededno
high-stakesexpert reviewneededno

Failure modes and limits

Model suitabilityhuman judgement required
Typical model failureReports "reviewed" when no reviewer saw the output.
Error sourcesinput data; coder or rater judgement; expert opinion
Certainty limitsCertainty drops when reviewers see only approved items (risk of bias), when review is rushed (imprecision) and when reviewers lack subject knowledge (indirectness).

Evidence and tag basis

Documented. NIST says reviewers should not be directly involved in developing the same model and that review should be prioritised by risk. The Australian standard asks for a competent person with authority to intervene. (NIST AI 600-1; Voluntary AI Safety Standard, 10 guardrails)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.