CH-03 Devil's advocacy
Builds the strongest case against the dominant view or plan, starting from its most vulnerable assumptions, and produces a labelled devil advocate finding.
What it does
Builds the strongest case against the dominant view or plan, starting from its most vulnerable assumptions. Artefact: findings presented to the group, labelled as devil's advocate product; read the vulnerable assumptions the advocate could not defend.
When to use it
Use when a view or plan is accepted too easily and you want the best case against it before a decision.
When it does not fit
The advocate is the plan's author (inference), or challenges are not taken seriously, which gives false confidence.
Result format
The result is a set of findings that make the strongest case against the dominant view or plan, starting from its most vulnerable assumptions and labelled as a devil's advocate product. It names the vulnerable assumptions that the advocate could not defend.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Unit of analysis | Always applies | requires=claim |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Content type, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | The mainline judgement and its key assumptions. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | The advocate is the plan author, or challenges are not taken seriously, which gives false confidence. |
Choosing and ordering
| Kind of technique | Challenge technique, runs in step 6 |
|---|---|
| Phase tag | validate |
| Other phase tags | frame; conclude |
| Kind of finding | risk; contradiction |
| Question answered | what; why |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to analyse |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | assumption and limitation statement | none | no |
| decision-informing | assumption and limitation statement | none | no |
| decision-informing | independent review | needed; left out below high stakes | no |
| high-stakes | assumption and limitation statement | none | no |
| high-stakes | independent review | needed | no |
| high-stakes | expert review | needed | no |
| high-stakes | second technique with different assumptions | none | conditional |
Failure modes and limits
| Model suitability | model alone |
|---|---|
| Typical model failure | Produces a mild objection that the plan already answers, or argues a position the model itself drafted. |
| Error sources | input data; coder or rater judgement; language model |
| Certainty limits | The CIA primer notes it takes analytic time; false confidence results when the challenge is not taken seriously, and there is no controlled evidence. |
Related techniques
- should follow CH-01 Key assumptions check (assumptions check): The most vulnerable assumptions found there are the starting points of the case against
- pairs with CH-11 Steelmanning Also serves: Compare.: Steelmanning strengthens the opposing position that the advocate argues from
Evidence and tag basis
Documented. The CIA primer notes it takes analytic time; the UK handbook pairs it with overconfidence; no controlled evidence found. (CIA Tradecraft Primer (2009); UK Red Teaming Handbook)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.