startthat.work home

MM-02 Algorithmic outlier and anomaly detection (Isolation Forest; Local Outlier Factor; Elliptic Envelope; One-Class SVM (support vector machine))

Score each record by how far it differs from the rest, using distance, density, isolation or a covariance estimate that resists outliers, and produce an outlier score and label per record.

Measure and modelEvidence: DocumentedPreparation technique, runs in step 5.2Strongest statement: describe

What it does

Scores records by how far they differ from the rest, using distance, density, isolation or a covariance estimate that resists outliers. Artefact: an outlier label or score per record. Read the top-scored records and decide whether they are errors, rare valid cases or a separate group.

When to use it

Use when you suspect errors, rare cases or hidden subgroups among many records and want them ranked for review.

When it does not fit

Choose by data shape: Elliptic Envelope for roughly Gaussian low-dimensional data, Isolation Forest for high-dimensional data with few assumptions, Local Outlier Factor when density varies. Misfit: flagged points form a dense cluster. Deleting flagged points without checking them is the common misuse.

Result format

The result is an outlier label or score for each record, with the top-scored records listed. Each of those records is given a decision: error, rare valid case or separate group.

Requirements on the material

PropertyRequirement typeRequirement
StructureApplies in some casesrequires=structured
Content typeAlways appliesrequires=tabular
Unit of analysisAlways appliesrequires=record
Variable typesRaises a warningprefers=interval; prefers=ratio
Level of detail availableAlways appliesrequires=record-level
Appraises source credibilityApplies in some casesno

Not restricted on: Time structure, How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialat least 100 records and two or more numeric variables for the multivariate methods
Additional sources needednone
Not for (rules)none
Not for (text)Wrong choice when the flagged points form a dense cluster, which suggests a subgroup and not outliers. Deleting flagged points without checking them is the common misuse.

Choosing and ordering

Kind of techniquePreparation technique, runs in step 5.2
Phase tagverify and prepare
Other phase tagsanalyse
Kind of findingpattern; risk
Question answeredwhat; who
Strongest statement it supportsdescribe
Needs as input
Makes as output
Must exist firstNone
Loop targetreturn to analyse

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: repeatable with fixed settings.

StakesCheckPerson involvedCounts as a different method
exploratoryinput validationnoneno
exploratoryrepeat runsnoneno
decision-informinginput validationnoneno
decision-informingrepeat runsnoneno
decision-informingsensitivity analysisnoneno
decision-informingspot checkoptionalno
high-stakesinput validationnoneno
high-stakesrepeat runsnoneno
high-stakessensitivity analysisnoneno
high-stakesspot checkoptionalno
high-stakessecond technique with different assumptionsnoneconditional
high-stakesindependent reviewneededno

Failure modes and limits

Model suitabilitymodel plus code
Typical model failureNames outliers by eye from a few rows, or picks a threshold to produce a desired count.
Error sourcesinput data; data transformation; statistical assumptions
Certainty limitsCertainty drops when the choice of algorithm or threshold changes the flagged set (inconsistency) and when no flagged record is checked against the source (risk of bias).

Evidence and tag basis

Documented. One-Class SVM is sensitive to outliers in the training data. (scikit-learn outlier detection)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.