TC1-15 Bayesian updating (likelihood reasoning, likelihood ratio, Bayes factor)
Revises the probability of a hypothesis in proportion to how much more likely the evidence is under it than under alternatives and produces a posterior probability or odds.
What it does
Revises the probability of a hypothesis in proportion to how much more likely the evidence is under it than under the alternatives. Artefact: a posterior probability or odds; read the likelihood ratio as the strength of the evidence.
When to use it
Use when you hold a prior view on a hypothesis and need to say how far a piece of evidence should move it.
When it does not fit
The prior or the likelihoods are guesses presented as precise numbers.
Result format
The result gives a prior probability for the hypothesis, a likelihood of each evidence item under the hypothesis and under the alternatives, and the likelihood ratio for each item. It ends with a posterior probability or odds, with the likelihood ratio read as the strength of the evidence.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Content type | Always applies | requires=text; requires=tabular |
| Unit of analysis | Always applies | requires=claim |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | A prior probability and the probability of each evidence item under each hypothesis. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Wrong choice when the prior or the likelihoods are guesses presented as precise numbers. Treating dependent evidence items as independent overstates the posterior. |
Choosing and ordering
| Kind of technique | Primary technique, runs in step 5.3 |
|---|---|
| Phase tag | analyse |
| Other phase tags | conclude |
| Kind of finding | cause; option ranking |
| Question answered | how much; why |
| Strongest statement it supports | predict, without claiming a cause |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | proceed with limit stated |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | assumption and limitation statement | none | no |
| decision-informing | assumption and limitation statement | none | no |
| decision-informing | sensitivity analysis | none | no |
| decision-informing | code execution for arithmetic | none | no |
| decision-informing | alternative explanations check | none | no |
| high-stakes | assumption and limitation statement | none | no |
| high-stakes | sensitivity analysis | none | no |
| high-stakes | code execution for arithmetic | none | no |
| high-stakes | alternative explanations check | none | no |
| high-stakes | second technique with different assumptions | none | conditional |
| high-stakes | expert review | needed | no |
| high-stakes | independent review | needed | no |
Failure modes and limits
| Model suitability | human judgement required |
|---|---|
| Typical model failure | States posterior probabilities to false precision from invented priors and likelihoods, or multiplies likelihoods of dependent evidence. |
| Error sources | input data; expert opinion; statistical assumptions |
| Certainty limits | Certainty is limited by the quality of the prior and the likelihoods, which are usually judgements. It drops further when evidence items are dependent or when the hypothesis set is incomplete (imprecision). |
Related techniques
- pairs with HE1-21 Distribution, outlier and base-rate checks: a base-rate check supplies defensible priors
- substitutes for TC1-13 Analysis of competing hypotheses (ACH): when evidence items can be given likelihoods as numbers (variable_types=ratio|counts or fractions)
Evidence and tag basis
Unconfirmed. No source was opened in this research.
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.