PE-05 Missing data diagnosis and treatment (missingness analysis; imputation; complete case analysis; non-response weighting)
Measure how much is missing, where and for whom, then choose deletion, imputation or weighting, giving the missing share by field and subgroup and a cleaned dataset.
What it does
Measures how much is missing, where and for whom, then chooses deletion, imputation or weighting. Artefact: the missing share by field and subgroup, and a comparison of complete and incomplete cases. Read for systematic patterns.
When to use it
Use when records have blank or absent values and you must decide how to treat them before analysis.
When it does not fit
Fits when the missing share is similar across subgroups and unrelated to key variables. Misfit: missingness differs by outcome or group, so deletion biases results. The mechanism needs a person. The classification into missing completely at random, at random and not at random was not confirmed from a source.
Result format
The result is a table of the missing share by field and by subgroup, and a comparison of complete and incomplete cases. It names the fields where missingness follows a systematic pattern and states the treatment chosen: deletion, imputation or weighting.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Structure | Applies in some cases | requires=structured; requires=semi-structured |
| Content type | Always applies | requires=tabular |
| Unit of analysis | Always applies | requires=record |
| Level of detail available | Always applies | requires=record-level |
| Known data problems | Applies in some cases | requires=missing data; requires=unknown |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Variable types, Time structure, How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | none |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Missingness differs by outcome or group so deletion would bias results, or no person can judge why values are missing. |
Choosing and ordering
| Kind of technique | Preparation technique, runs in step 5.2 |
|---|---|
| Phase tag | verify and prepare |
| Other phase tags | none |
| Kind of finding | gap; risk |
| Question answered | what; how much |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to gather |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | input validation | none | no |
| decision-informing | input validation | none | no |
| decision-informing | sensitivity analysis | none | no |
| decision-informing | spot check | optional | no |
| high-stakes | input validation | none | no |
| high-stakes | sensitivity analysis | none | no |
| high-stakes | spot check | optional | no |
| high-stakes | second technique with different assumptions | none | conditional |
| high-stakes | provenance log | none | no |
| high-stakes | expert review | needed | no |
Failure modes and limits
| Model suitability | human judgement required |
|---|---|
| Typical model failure | Drops incomplete records or imputes means without testing whether missingness is related to the outcome. |
| Error sources | input data; data transformation; statistical assumptions |
| Certainty limits | Certainty drops when the missingness mechanism cannot be judged (indirectness), when complete and incomplete cases differ (risk of bias), or when results change between treatments (inconsistency). |
Related techniques
- should follow PE-04 Data profiling and rule-based validation (data audit; data quality assessment; edit checks): profiling locates the missing values and their patterns
- pairs with MM-06 Survey weighting and subgroup estimation (design weights; calibration; post-stratification; relative standard error): survey weighting is the alternative treatment for non-response in survey records
Evidence and tag basis
Documented. The ABS states that imputation and weighting are the two standard treatments of non-response. (ABS Methodological News, Dec 2009)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.