MM-04 Dimensionality reduction (PCA (principal component analysis); factor analysis; non-negative matrix factorisation)
Replace many correlated numeric fields with a few components and produce loadings, explained variance and a score per record.
What it does
Replaces many correlated numeric fields with a few components that capture most of the variance. Artefact: loadings, explained variance ratio and scores per record. Read loadings to name components and explained variance to choose their number.
When to use it
Use when there are many correlated numeric variables and you want a smaller set of dimensions to read or model.
When it does not fit
Fits correlated fields with linear structure. PCA centres but does not scale, so large-variance fields dominate unless standardised. Misfit: the first components explain little variance, or loadings cannot be interpreted.
Result format
The result has a table of loadings for each component, the explained variance ratio for each component, and a score for each record. It names each component from its loadings and states how many components were kept and why.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Structure | Applies in some cases | requires=structured |
| Content type | Always applies | requires=tabular |
| Unit of analysis | Always applies | requires=variable; requires=dataset |
| Variable types | Raises a warning | prefers=interval; prefers=ratio |
| Level of detail available | Always applies | requires=record-level |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Time structure, How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | at least 3 numeric variables and more records than variables |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Wrong choice when the first components explain little variance or the loadings cannot be interpreted. Non-negative matrix factorisation also needs non-negative data. |
Choosing and ordering
| Kind of technique | Primary technique, runs in step 5.3 |
|---|---|
| Phase tag | analyse |
| Other phase tags | verify and prepare |
| Kind of finding | pattern |
| Question answered | what |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to analyse |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: deterministic.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | input validation | none | no |
| decision-informing | input validation | none | no |
| decision-informing | sensitivity analysis | none | no |
| decision-informing | spot check | optional | no |
| high-stakes | input validation | none | no |
| high-stakes | sensitivity analysis | none | no |
| high-stakes | spot check | optional | no |
| high-stakes | independent review | needed | no |
Failure modes and limits
| Model suitability | human judgement required |
|---|---|
| Typical model failure | Invents plausible names for components without checking loadings. |
| Error sources | input data; data transformation; statistical assumptions |
| Certainty limits | Certainty drops when variables are not standardised (risk of bias), when components are named without reading loadings (indirectness) and when results change with the number of components kept (inconsistency). |
Related techniques
- should follow PE-04 Data profiling and rule-based validation (data audit; data quality assessment; edit checks): unit and scale problems distort the components
- pairs with MM-03 Clustering of records (k-means; hierarchical (agglomerative) clustering; DBSCAN (density-based spatial clustering of applications with noise); HDBSCAN (hierarchical DBSCAN)): components can be used as input to clustering of records
Evidence and tag basis
Documented. Non-negative matrix factorisation needs non-negative data. (scikit-learn decomposition)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.