PE-08 Named entity recognition (NER; entity extraction)
Tag spans of text as people, organisations, places, dates and other types, giving a table of mentions with type and position and a tagged corpus.
What it does
Tags spans of text as people, organisations, places, dates and other types. Artefact: a table of mentions with type and position. Read which actors recur and in what contexts.
When to use it
Use when you need to know which people, organisations or places are named in a body of text and in what contexts.
When it does not fit
Entity types must match the question. Domain terms, such as company names in filings or agency names in policy documents, are often mistagged. No domain accuracy figures were verified.
Result format
The result is a table of mentions, one row per mention, giving the text span, the entity type and the position. It ends with the actors that recur and the contexts in which they appear.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Structure | Applies in some cases | requires=unstructured; requires=semi-structured |
| Content type | Always applies | requires=text |
| Unit of analysis | Always applies | requires=document |
| Appraises source credibility | Applies in some cases | no |
| Personal information | Always applies | requires=no personal information; requires=personal information |
Not restricted on: Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Security classification, Licence.
| Minimum material | A tagger or model for the language of the text, and entity types that match the question. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Domain terms such as company names in filings or agency names in policy documents are likely to be mistagged and the entity types do not fit the question. |
Choosing and ordering
| Kind of technique | Preparation technique, runs in step 5.2 |
|---|---|
| Phase tag | verify and prepare |
| Other phase tags | analyse |
| Kind of finding | pattern |
| Question answered | who; what |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to verify and prepare |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: repeatable with fixed settings.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | spot check | optional | no |
| decision-informing | spot check | optional | no |
| decision-informing | human review sample | needed; left out below high stakes | no |
| decision-informing | gold standard agreement | conditional; left out below high stakes | no |
| high-stakes | spot check | optional | no |
| high-stakes | human review sample | needed | no |
| high-stakes | gold standard agreement | conditional | no |
| high-stakes | second technique with different assumptions | none | conditional |
| high-stakes | independent review | needed | no |
| high-stakes | provenance log | none | no |
Failure modes and limits
| Model suitability | model plus code |
|---|---|
| Typical model failure | Tags domain terms with the wrong type and treats each distinct spelling of a name as a separate entity. |
| Error sources | input data; statistical assumptions; language model |
| Certainty limits | Certainty drops when accuracy has not been measured for the domain (indirectness), when entity types do not fit the question (indirectness), or when names vary (inconsistency). |
Related techniques
- pairs with IS-01 Entity resolution and linking (record linkage; entity matching; entity linking; coreference resolution; blocking): entity resolution joins mentions of one entity that appear under different names
- pairs with CK-04 Gold-standard validation sample (ground truth set; human-coded validation set): a gold-standard sample measures tagging accuracy for the domain
- should follow PE-03 Near-duplicate detection (shingling; MinHash; locality-sensitive hashing; text reuse detection): removing repeated documents stops one mention being counted many times
Evidence and tag basis
Documented. A textbook chapter on sequence labelling for named entities is listed on the contents page; the chapter was not opened. (Jurafsky and Martin, SLP3 contents)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.