HE1-15 Content analysis (conventional, directed and summative)
Codes text into categories, then counts or interprets category use, and produces category frequencies or a category structure.
What it does
Codes text into categories, then counts or interprets category use. Artefact: category frequencies or a category structure; read agreement between coders where used.
When to use it
Use when you need to say how often topics or categories appear in a body of text.
When it does not fit
Meaning depends on context that categories remove.
Result format
The result is a category structure or a table of category frequencies, with one row per category and a count or interpretation of how often the category is used. It reports agreement between coders where more than one coder was used.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Content type | Always applies | requires=text |
| Unit of analysis | Always applies | requires=document |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | a coding scheme |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Meaning depends on context that the categories remove. |
Choosing and ordering
| Kind of technique | Primary technique, runs in step 5.3 |
|---|---|
| Phase tag | analyse |
| Other phase tags | none |
| Kind of finding | pattern; trend |
| Question answered | what; how much |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | proceed with limit stated |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: repeatable with fixed settings.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | spot check | optional | no |
| decision-informing | spot check | optional | no |
| decision-informing | repeat runs | none | no |
| decision-informing | human review sample | needed; left out below high stakes | no |
| high-stakes | spot check | optional | no |
| high-stakes | repeat runs | none | no |
| high-stakes | human review sample | needed | no |
| high-stakes | second technique with different assumptions | none | conditional |
| high-stakes | independent review | needed | no |
Failure modes and limits
| Model suitability | model plus code |
|---|---|
| Typical model failure | Applies categories inconsistently across documents and reports counts without any agreement check. |
| Error sources | input data; coder or rater judgement; language model |
| Certainty limits | Certainty drops with low coder agreement, a corpus that was not sampled for the question (indirectness), and category drift. |
Related techniques
- pairs with CK-01 Inter-coder agreement statistics (percentage agreement; Cohen's kappa; weighted kappa; Scott's pi; Fleiss' kappa; Krippendorff's alpha; intercoder reliability testing): agreement between coders tests whether the categories are applied consistently
- substitutes for MM-08 LLM coding and zero-shot classification of text (LLM annotation; LLM-assisted deductive coding; few-shot classification; natural language inference classification): when volume_class is readable by a person and manual coding is feasible
Evidence and tag basis
Unconfirmed. No source was opened.
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.