startthat.work home

HE1-03 CRAAP test (currency, relevance, authority, accuracy, purpose)

Screens one source against five criteria (currency, relevance, authority, accuracy, purpose) and produces a checklist judgement for the evidence base.

Handle evidenceEvidence: DocumentedPreparation technique, runs in step 5.2Strongest statement: describe

What it does

Screens a single source against five criteria. Artefact: a checklist judgement; read authority, accuracy and purpose first for vendor material.

When to use it

Use when you need a quick check on whether a web page, report or vendor document is fit to cite.

When it does not fit

It is used to judge a specific claim, because it examines the page, not the claim's wider support. The URL domain heuristic is weak (note author's judgement).

Result format

The result is a checklist with one row per criterion (currency, relevance, authority, accuracy, purpose), each giving a judgement and the reason for it. It ends with an overall judgement of the single source, with the authority, accuracy and purpose rows stated first when the source is vendor material.

Requirements on the material

PropertyRequirement typeRequirement
Content typeAlways appliesrequires=text
Unit of analysisAlways appliesrequires=source; requires=document
Producer interestChanges the orderprefers=sponsored; prefers=self-interested seller; prefers=unknown
OriginChanges the orderprefers=secondary; prefers=tertiary; prefers=unknown
Appraises source credibilityApplies in some casesyes

Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Source dependence, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialnone
Additional sources needednone
Not for (rules)none
Not for (text)The target is a specific claim and not the page, because the test does not examine the claim's wider support. A domain name used as a proxy for authority is weak.

Choosing and ordering

Kind of techniquePreparation technique, runs in step 5.2
Phase tagverify and prepare
Other phase tagsnone
Kind of findingrisk
Question answeredwhat; who
Strongest statement it supportsdescribe
Needs as input
Makes as output
Must exist firstNone
Loop targetproceed with limit stated

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.

StakesCheckPerson involvedCounts as a different method
exploratoryspot checkoptionalno
decision-informingspot checkoptionalno
decision-informingrepeat runsnoneno
decision-informingprompt variationnoneno
high-stakesspot checkoptionalno
high-stakesrepeat runsnoneno
high-stakesprompt variationnoneno
high-stakeshuman review sampleneededno
high-stakesindependent reviewneededno

Failure modes and limits

Model suitabilitymodel alone
Typical model failureInfers authority from how professional the page looks, and invents author credentials not shown on the page.
Error sourcesinput data; language model; coder or rater judgement
Certainty limitsCertainty drops when the criteria are scored without a rule, when authority cannot be checked outside the page, or when purpose is hidden (risk of bias).

Evidence and tag basis

Documented. A library handout with criteria and questions; its creator is not named on it. (Meriam Library CRAAP test handout)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.