startthat.work home

PE-04 Data profiling and rule-based validation (data audit; data quality assessment; edit checks)

Describe the structure, completeness, validity and consistency of each field and test explicit rules, giving a profile table per column and a list of records that break each rule.

Prepare and extractEvidence: DocumentedPreparation technique, runs in step 5.2Strongest statement: describe

What it does

Describes the structure, completeness, validity and consistency of each field and tests explicit rules such as end date after start date. Artefact: a profile table per column and a list of records that break each rule. Read for missing patterns, impossible values, duplicate keys and inconsistent codes.

When to use it

Use when you have received a table or dataset and need to know its quality before any analysis.

When it does not fit

Always applicable as a first step. Profiling shows that a value is odd, not that it is wrong. The analyst chooses the rule set, and it is never complete.

Result format

The result has a profile table with one row per column, giving its type, completeness, valid range and consistency, and a list of the records that break each stated rule. It ends by naming the missing patterns, impossible values, duplicate keys and inconsistent codes found.

Requirements on the material

PropertyRequirement typeRequirement
StructureApplies in some casesrequires=structured; requires=semi-structured
Content typeAlways appliesrequires=tabular
Unit of analysisAlways appliesrequires=dataset; requires=variable
Level of detail availableAlways appliesrequires=record-level
Appraises source credibilityApplies in some casesno

Not restricted on: Variable types, Time structure, How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialnone
Additional sources neededreference structure
Not for (rules)none
Not for (text)The profile is read as proof that a value is wrong; it shows only that a value is unusual, and the rule set chosen by the analyst is never complete.

Choosing and ordering

Kind of techniquePreparation technique, runs in step 5.2
Phase tagverify and prepare
Other phase tagsnone
Kind of findinggap; contradiction; risk
Question answeredwhat
Strongest statement it supportsdescribe
Needs as input
Makes as output
Must exist firstNone
Loop targetreturn to gather

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: deterministic.

StakesCheckPerson involvedCounts as a different method
exploratoryinput validationnoneno
decision-informinginput validationnoneno
decision-informingspot checkoptionalno
decision-informingprovenance lognoneno
high-stakesinput validationnoneno
high-stakesspot checkoptionalno
high-stakesprovenance lognoneno
high-stakeshuman review sampleneededno
high-stakesindependent reviewneededno

Failure modes and limits

Model suitabilitymodel plus code
Typical model failureReports that odd values are errors and writes a short rule set as if it were complete.
Error sourcesinput data; data transformation
Certainty limitsCertainty about quality drops when the rule set is short (reporting bias) or when field definitions are undocumented (indirectness).

Evidence and tag basis

Documented. The Australian Bureau of Statistics (ABS) framework defines seven quality dimensions. RADAR names five artefact classes: missing data, bad values, outliers, inconsistent formatting and inconsistent logic. The column profile layout is practice with no authoritative definition. (ABS data quality framework; RADAR, arXiv 2506.08249)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.