PJ-21 Robust decision making, adaptive pathways and info-gap analysis
Test many model runs or futures to find the conditions under which a strategy fails and design strategies that adapt, giving vulnerability scenarios and pathway maps with trigger points as a draft result.
What it does
Tests many model runs or futures to find the conditions under which a strategy fails, then designs strategies that adapt. Artefact: vulnerability scenarios and pathway maps with trigger points.
When to use it
Use when you have a simulation model and a decision under deep uncertainty and need strategies that adapt as conditions change.
When it does not fit
No quantitative performance model exists, or trigger points are not tied to something that can be monitored.
Result format
The result lists the vulnerability scenarios, meaning the conditions under which each strategy fails, and a pathway map for the adapted strategies. Each pathway names its trigger points.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Content type | Always applies | requires=tabular |
| Unit of analysis | Always applies | requires=topic |
| Level of detail available | Always applies | requires=aggregate tables |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | A simulation model or explicit uncertain parameters, candidate strategies and performance thresholds. |
|---|---|
| Additional sources needed | reference structure |
| Not for (rules) | none |
| Not for (text) | No quantitative performance model exists, or trigger points are not tied to something that can be monitored. |
Choosing and ordering
| Kind of technique | Primary technique, runs in step 5.3 |
|---|---|
| Phase tag | analyse |
| Other phase tags | validate |
| Kind of finding | risk; option ranking |
| Question answered | which option; what next |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to analyse |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: repeatable with fixed settings.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | input validation | none | no |
| decision-informing | input validation | none | no |
| decision-informing | sensitivity analysis | none | no |
| decision-informing | spot check | optional | no |
| high-stakes | input validation | none | no |
| high-stakes | sensitivity analysis | none | no |
| high-stakes | spot check | optional | no |
| high-stakes | second technique with different assumptions | none | conditional |
| high-stakes | expert review | needed | no |
| high-stakes | assumption and limitation statement | none | no |
Failure modes and limits
| Model suitability | model plus code |
|---|---|
| Typical model failure | Defines trigger points that no available indicator can measure. |
| Error sources | input data; statistical assumptions; expert opinion |
| Certainty limits | Confidence drops when the performance model is unvalidated (indirectness), thresholds are set by one group (risk of bias), or the technique source was not confirmed. |
Related techniques
- pairs with PJ-19 Policy stress-testing: a table of performance by condition is a lighter alternative when no model exists
- pairs with PJ-25 Leading indicators and signposts (indicators of change, trigger points): trigger points need indicators that can be monitored
Evidence and tag basis
Unconfirmed. Attributed to the RAND Corporation and to Haasnoot and colleagues from memory; not opened.
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.