MM-02 Algorithmic outlier and anomaly detection (Isolation Forest; Local Outlier Factor; Elliptic Envelope; One-Class SVM (support vector machine))
Score each record by how far it differs from the rest, using distance, density, isolation or a covariance estimate that resists outliers, and produce an outlier score and label per record.
What it does
Scores records by how far they differ from the rest, using distance, density, isolation or a covariance estimate that resists outliers. Artefact: an outlier label or score per record. Read the top-scored records and decide whether they are errors, rare valid cases or a separate group.
When to use it
Use when you suspect errors, rare cases or hidden subgroups among many records and want them ranked for review.
When it does not fit
Choose by data shape: Elliptic Envelope for roughly Gaussian low-dimensional data, Isolation Forest for high-dimensional data with few assumptions, Local Outlier Factor when density varies. Misfit: flagged points form a dense cluster. Deleting flagged points without checking them is the common misuse.
Result format
The result is an outlier label or score for each record, with the top-scored records listed. Each of those records is given a decision: error, rare valid case or separate group.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Structure | Applies in some cases | requires=structured |
| Content type | Always applies | requires=tabular |
| Unit of analysis | Always applies | requires=record |
| Variable types | Raises a warning | prefers=interval; prefers=ratio |
| Level of detail available | Always applies | requires=record-level |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Time structure, How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | at least 100 records and two or more numeric variables for the multivariate methods |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Wrong choice when the flagged points form a dense cluster, which suggests a subgroup and not outliers. Deleting flagged points without checking them is the common misuse. |
Choosing and ordering
| Kind of technique | Preparation technique, runs in step 5.2 |
|---|---|
| Phase tag | verify and prepare |
| Other phase tags | analyse |
| Kind of finding | pattern; risk |
| Question answered | what; who |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to analyse |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: repeatable with fixed settings.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | input validation | none | no |
| exploratory | repeat runs | none | no |
| decision-informing | input validation | none | no |
| decision-informing | repeat runs | none | no |
| decision-informing | sensitivity analysis | none | no |
| decision-informing | spot check | optional | no |
| high-stakes | input validation | none | no |
| high-stakes | repeat runs | none | no |
| high-stakes | sensitivity analysis | none | no |
| high-stakes | spot check | optional | no |
| high-stakes | second technique with different assumptions | none | conditional |
| high-stakes | independent review | needed | no |
Failure modes and limits
| Model suitability | model plus code |
|---|---|
| Typical model failure | Names outliers by eye from a few rows, or picks a threshold to produce a desired count. |
| Error sources | input data; data transformation; statistical assumptions |
| Certainty limits | Certainty drops when the choice of algorithm or threshold changes the flagged set (inconsistency) and when no flagged record is checked against the source (risk of bias). |
Related techniques
- should follow PE-04 Data profiling and rule-based validation (data audit; data quality assessment; edit checks): rule-based checks remove impossible values so the algorithm scores real anomalies
- pairs with MM-01 Exploratory data analysis with plots and distribution checks (EDA (exploratory data analysis); summary statistics; Q-Q plot; scatterplot matrix): plots show whether flagged records are errors or a separate group
Evidence and tag basis
Documented. One-Class SVM is sensitive to outliers in the training data. (scikit-learn outlier detection)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.