startthat.work home

IS-01 Entity resolution and linking (record linkage; entity matching; entity linking; coreference resolution; blocking)

Decides which records or text mentions refer to the same entity by comparing attributes, with no shared identifier required, and produces match decisions or linked clusters.

Integrate across sourcesEvidence: DocumentedPreparation technique, runs in step 5.2Strongest statement: describe

What it does

Decides which records or text mentions refer to the same entity by comparing attributes, with no shared identifier required. Steps: pre-processing, indexing (blocking), comparison, classification and evaluation. Artefact: match decisions or linked clusters. Read the match count, the uncertain pairs and the unresolved rate before counting actors.

When to use it

Use when the same people, organisations or places appear in different sources under different names or spellings and you need one list.

When it does not fit

Fits when records share enough comparable attributes. Misfit: sparse, changing or unreliable attributes. Blocking needs schema knowledge unless a schema-agnostic method is used. No accuracy figures for matching without identifiers were found.

Result format

The result is a set of match decisions or linked clusters, with the match count, the list of uncertain pairs and the unresolved rate. It gives these figures before any count of actors.

Requirements on the material

PropertyRequirement typeRequirement
Content typeAlways appliesrequires=text; requires=tabular
Unit of analysisAlways appliesrequires=record
Level of detail availableAlways appliesrequires=record-level
Number of sourcesAlways appliesrequires=two or more
Whether sources can be joinedAlways appliesrequires=no shared identifier; requires=shared identifier available
Appraises source credibilityApplies in some casesno
Personal informationAlways appliesrequires=no personal information; requires=personal information

Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Security classification, Licence.

Minimum materialrecords that share enough comparable attributes
Additional sources needednone
Not for (rules)none
Not for (text)Attributes are sparse, changing or unreliable. Blocking needs schema knowledge unless a schema-agnostic method is used, and no accuracy figures for matching without identifiers were found.

Choosing and ordering

Kind of techniquePreparation technique, runs in step 5.2
Phase tagverify and prepare
Other phase tagsanalyse
Kind of findingpattern; gap
Question answeredwho; what
Strongest statement it supportsdescribe
Needs as input
Makes as output
Must exist firstNone
Loop targetproceed with limit stated

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: repeatable with fixed settings.

StakesCheckPerson involvedCounts as a different method
exploratoryspot checkoptionalno
decision-informingspot checkoptionalno
decision-informingrepeat runsnoneno
decision-informinghuman review sampleneeded; left out below high stakesno
high-stakesspot checkoptionalno
high-stakesrepeat runsnoneno
high-stakeshuman review sampleneededno
high-stakessecond technique with different assumptionsnoneconditional
high-stakesindependent reviewneededno

Failure modes and limits

Model suitabilitymodel plus code
Typical model failureMerges different entities with similar names, and does not report the uncertain pairs or the unresolved rate.
Error sourcesinput data; data transformation; statistical assumptions
Certainty limitsCertainty drops with a high share of uncertain pairs, no labelled sample to measure match accuracy, and attributes that change over time (imprecision and risk of bias).

Evidence and tag basis

Documented. A publisher note says an off-the-shelf matching system usually needs substantial adaptation. Only a book description, a survey abstract and a textbook contents page were read. (Christen, Data Matching, Springer book page; Papadakis et al., arXiv 1905.06167; Jurafsky and Martin, SLP3 contents)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.