PJ-17 Cross-impact analysis (cross-impact matrix)
Adjust the probability of each event in a set according to how every other event raises or lowers it, giving adjusted event probabilities and an influence matrix as a draft result.
What it does
Adjusts the probability of each event in a set according to how every other event raises or lowers it. Artefact: adjusted event probabilities and a matrix of influential and affected variables.
When to use it
Use when you have 10 to 40 linked events and need to see how their probabilities change once the links between them are taken into account.
When it does not fit
Conditional probabilities are guessed, the judgements are inconsistent (impossible values can result), or variables are not events.
Result format
The result has a list of the events with their adjusted probabilities, and a matrix in which each cell shows how one event raises or lowers another. It names the influential variables and the affected variables.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Unit of analysis | Always applies | requires=topic |
| Variable types | Raises a warning | prefers=counts or fractions |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Content type, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | About 10 to 40 events, an initial probability for each and pairwise conditional probabilities, usually from expert opinion. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Conditional probabilities are guessed, the judgements are inconsistent so impossible values result, or variables are not events. |
Choosing and ordering
| Kind of technique | Primary technique, runs in step 5.3 |
|---|---|
| Phase tag | analyse |
| Other phase tags | none |
| Kind of finding | risk; pattern |
| Question answered | what next; how much |
| Strongest statement it supports | predict, without claiming a cause |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to gather |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | assumption and limitation statement | none | no |
| exploratory | input validation | none | no |
| decision-informing | assumption and limitation statement | none | no |
| decision-informing | input validation | none | no |
| decision-informing | code execution for arithmetic | none | no |
| decision-informing | sensitivity analysis | none | no |
| high-stakes | assumption and limitation statement | none | no |
| high-stakes | input validation | none | no |
| high-stakes | code execution for arithmetic | none | no |
| high-stakes | sensitivity analysis | none | no |
| high-stakes | independent review | needed | no |
| high-stakes | expert review | needed | no |
| high-stakes | second technique with different assumptions | none | conditional |
Failure modes and limits
| Model suitability | human judgement required |
|---|---|
| Typical model failure | Fills the conditional probability matrix with plausible guesses that give impossible joint values. |
| Error sources | expert opinion; statistical assumptions |
| Certainty limits | Certainty drops when judgements come from one group (risk of bias), cannot be reproduced by another group (inconsistency), or are guessed (imprecision). |
Related techniques
- should follow PJ-04 Delphi technique (real-time Delphi, mini-Delphi): a Delphi panel supplies the initial and conditional probabilities
- should follow PJ-06 Structured expert elicitation (IDEA protocol: investigate, discuss, estimate, aggregate): structured elicitation gives numeric ranges for the inputs
- pairs with TC1-17 Sensitivity analysis (robustness check, leave-one-out): sensitivity analysis shows which judgements move the result
Evidence and tag basis
Documented. The intelligence-analysis variant depends heavily on analysts' opinion and is hard to reproduce with a different group. (Wikipedia, Cross-impact analysis)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.