TC1-13 Analysis of competing hypotheses (ACH)
Lists plausible hypotheses, rates each evidence item against each, and produces a hypothesis by evidence matrix aimed at disproving hypotheses.
What it does
Lists all plausible hypotheses and rates each evidence item against each, aiming to disprove hypotheses. Artefact: a hypothesis by evidence matrix; read the evidence that discriminates between hypotheses and the hypothesis with the fewest inconsistencies.
When to use it
Use when several explanations are plausible and you need to see which evidence actually discriminates between them.
When it does not fit
The hypothesis list is incomplete, analysts skip steps, or non-diagnostic evidence shifts the result.
Result format
The result lists the hypotheses and gives a matrix of evidence items against hypotheses with a consistency rating in each cell. It names the evidence items that discriminate between hypotheses and the hypothesis with the fewest inconsistencies.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Content type | Always applies | requires=text |
| Unit of analysis | Always applies | requires=claim |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | At least two rival hypotheses and several evidence items rated against each. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Wrong choice when the hypothesis list is incomplete, analysts skip steps, or non-diagnostic evidence shifts the result. Experiments found no accuracy benefit and some harm to coherence. |
Choosing and ordering
| Kind of technique | Primary technique, runs in step 5.3 |
|---|---|
| Phase tag | analyse |
| Other phase tags | validate |
| Kind of finding | contradiction; cause; difference |
| Question answered | why; what |
| Strongest statement it supports | show a link between things |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to frame |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | assumption and limitation statement | none | no |
| decision-informing | assumption and limitation statement | none | no |
| decision-informing | alternative explanations check | none | no |
| decision-informing | source-span citation | none | no |
| high-stakes | assumption and limitation statement | none | no |
| high-stakes | alternative explanations check | none | no |
| high-stakes | source-span citation | none | no |
| high-stakes | second technique with different assumptions | none | conditional |
| high-stakes | expert review | needed | no |
| high-stakes | independent review | needed | no |
Failure modes and limits
| Model suitability | human judgement required |
|---|---|
| Typical model failure | Rates most evidence items as consistent with every hypothesis so that nothing discriminates, and omits the hypothesis the analyst finds unlikely. |
| Error sources | input data; source selection; coder or rater judgement |
| Certainty limits | Certainty drops when the hypothesis list is incomplete, when evidence is not diagnostic, or when all ratings come from one analyst (risk of bias and inconsistency). |
Related techniques
- pairs with TC1-16 Disconfirmation search (falsification, seeking disconfirming evidence): a disconfirmation search tests the hypothesis that survives the matrix
- substitutes for TC1-15 Bayesian updating (likelihood reasoning, likelihood ratio, Bayes factor): when evidence is available only as narrative and probabilities cannot be stated (access_depth=narrative only)
- pairs with CH-01 Key assumptions check (assumptions check): a key assumptions check exposes premises shared by every hypothesis
Evidence and tag basis
Tested. Tested in experiments with no accuracy benefit and some signs of harm to coherence; see the evidence section. Heuer concedes it will not always yield the right answer. (Dhami, Belton and Mandel (2019), Strathprints record; Karvetski and Mandel (2020), Cambridge Core, does ACH help coherence; Cambridge Core paper on what works and what fails in improving analysts' accuracy; Heuer, Psychology of Intelligence Analysis)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.