TC1-05 Counterfactual analysis (potential outcomes, minimal rewrite of history) Also serves: Compare.
Sets the actual outcome against an argued account of what would have happened had one factor differed and produces a counterfactual scenario.
What it does
Sets the actual outcome against an argued account of what would have happened had one factor differed. Artefact: a counterfactual scenario; read divergence from the actual outcome as the causal effect of the factor.
When to use it
Use when you must judge whether one factor caused an outcome in a single case and no comparison data exist.
When it does not fit
Varying the factor requires changing other factors that would also matter (low cotenability). Fewer required changes make the inference easier to defend; a 'but for' condition is not automatically a cause.
Result format
The result states the actual outcome, the one factor that is changed, and a counterfactual scenario describing what would have happened had that factor differed. It ends with the divergence between the scenario and the actual outcome, read as the causal effect of the factor.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Content type | Always applies | requires=text |
| Unit of analysis | Always applies | requires=claim |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | One specified change to a single factor, the relevant historical facts, and a stated theory or regularity linking the factor to the outcome. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Wrong choice when varying the factor requires changing other factors that would also matter (low cotenability). A but-for condition is not automatically a cause. |
Choosing and ordering
| Kind of technique | Primary technique, runs in step 5.3 |
|---|---|
| Phase tag | analyse |
| Other phase tags | none |
| Kind of finding | cause; difference |
| Question answered | why |
| Strongest statement it supports | explain how or why |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to analyse |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | assumption and limitation statement | none | no |
| decision-informing | assumption and limitation statement | none | no |
| decision-informing | alternative explanations check | none | no |
| high-stakes | assumption and limitation statement | none | no |
| high-stakes | alternative explanations check | none | no |
| high-stakes | second technique with different assumptions | none | conditional |
| high-stakes | expert review | needed | no |
| high-stakes | independent review | needed | no |
Failure modes and limits
| Model suitability | human judgement required |
|---|---|
| Typical model failure | Rewrites history so that several other factors change at once, then credits the divergence to the one factor named. |
| Error sources | input data; coder or rater judgement; expert opinion |
| Certainty limits | Certainty drops when the change needs other changes, when the theory linking factor to outcome is weak, or when the argued outcome cannot be checked against any evidence (indirectness). |
Related techniques
- substitutes for TC1-06 Difference-in-differences Also serves: Compare.: when no comparison group or before and after data exist (access_depth=narrative only)
- pairs with TC1-04 Process tracing (diagnostic tests: straw in the wind, hoop, smoking gun, doubly decisive): process tracing supplies the within-case evidence the counterfactual is checked against
Evidence and tag basis
Documented. Fearon argues counterfactuals are unavoidable in causal testing with few cases; the Stanford Encyclopedia lists vagueness of similarity and counterfactual scepticism as limits. (Fearon, Counterfactuals and Hypothesis Testing in Political Science; Stanford Encyclopedia of Philosophy, Counterfactuals)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.