startthat.work home

PE-05 Missing data diagnosis and treatment (missingness analysis; imputation; complete case analysis; non-response weighting)

Measure how much is missing, where and for whom, then choose deletion, imputation or weighting, giving the missing share by field and subgroup and a cleaned dataset.

Prepare and extractEvidence: DocumentedPreparation technique, runs in step 5.2Strongest statement: describe

What it does

Measures how much is missing, where and for whom, then chooses deletion, imputation or weighting. Artefact: the missing share by field and subgroup, and a comparison of complete and incomplete cases. Read for systematic patterns.

When to use it

Use when records have blank or absent values and you must decide how to treat them before analysis.

When it does not fit

Fits when the missing share is similar across subgroups and unrelated to key variables. Misfit: missingness differs by outcome or group, so deletion biases results. The mechanism needs a person. The classification into missing completely at random, at random and not at random was not confirmed from a source.

Result format

The result is a table of the missing share by field and by subgroup, and a comparison of complete and incomplete cases. It names the fields where missingness follows a systematic pattern and states the treatment chosen: deletion, imputation or weighting.

Requirements on the material

PropertyRequirement typeRequirement
StructureApplies in some casesrequires=structured; requires=semi-structured
Content typeAlways appliesrequires=tabular
Unit of analysisAlways appliesrequires=record
Level of detail availableAlways appliesrequires=record-level
Known data problemsApplies in some casesrequires=missing data; requires=unknown
Appraises source credibilityApplies in some casesno

Not restricted on: Variable types, Time structure, How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialnone
Additional sources needednone
Not for (rules)none
Not for (text)Missingness differs by outcome or group so deletion would bias results, or no person can judge why values are missing.

Choosing and ordering

Kind of techniquePreparation technique, runs in step 5.2
Phase tagverify and prepare
Other phase tagsnone
Kind of findinggap; risk
Question answeredwhat; how much
Strongest statement it supportsdescribe
Needs as input
Makes as output
Must exist firstNone
Loop targetreturn to gather

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.

StakesCheckPerson involvedCounts as a different method
exploratoryinput validationnoneno
decision-informinginput validationnoneno
decision-informingsensitivity analysisnoneno
decision-informingspot checkoptionalno
high-stakesinput validationnoneno
high-stakessensitivity analysisnoneno
high-stakesspot checkoptionalno
high-stakessecond technique with different assumptionsnoneconditional
high-stakesprovenance lognoneno
high-stakesexpert reviewneededno

Failure modes and limits

Model suitabilityhuman judgement required
Typical model failureDrops incomplete records or imputes means without testing whether missingness is related to the outcome.
Error sourcesinput data; data transformation; statistical assumptions
Certainty limitsCertainty drops when the missingness mechanism cannot be judged (indirectness), when complete and incomplete cases differ (risk of bias), or when results change between treatments (inconsistency).

Evidence and tag basis

Documented. The ABS states that imputation and weighting are the two standard treatments of non-response. (ABS Methodological News, Dec 2009)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.