startthat.work home

PJ-17 Cross-impact analysis (cross-impact matrix)

Adjust the probability of each event in a set according to how every other event raises or lowers it, giving adjusted event probabilities and an influence matrix as a draft result.

Project forwardEvidence: DocumentedPrimary technique, runs in step 5.3Strongest statement: predict, without claiming a cause

What it does

Adjusts the probability of each event in a set according to how every other event raises or lowers it. Artefact: adjusted event probabilities and a matrix of influential and affected variables.

When to use it

Use when you have 10 to 40 linked events and need to see how their probabilities change once the links between them are taken into account.

When it does not fit

Conditional probabilities are guessed, the judgements are inconsistent (impossible values can result), or variables are not events.

Result format

The result has a list of the events with their adjusted probabilities, and a matrix in which each cell shows how one event raises or lowers another. It names the influential variables and the affected variables.

Requirements on the material

PropertyRequirement typeRequirement
Unit of analysisAlways appliesrequires=topic
Variable typesRaises a warningprefers=counts or fractions
Appraises source credibilityApplies in some casesno

Not restricted on: Structure, Content type, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialAbout 10 to 40 events, an initial probability for each and pairwise conditional probabilities, usually from expert opinion.
Additional sources needednone
Not for (rules)none
Not for (text)Conditional probabilities are guessed, the judgements are inconsistent so impossible values result, or variables are not events.

Choosing and ordering

Kind of techniquePrimary technique, runs in step 5.3
Phase taganalyse
Other phase tagsnone
Kind of findingrisk; pattern
Question answeredwhat next; how much
Strongest statement it supportspredict, without claiming a cause
Needs as input
Makes as output
Must exist firstNone
Loop targetreturn to gather

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.

StakesCheckPerson involvedCounts as a different method
exploratoryassumption and limitation statementnoneno
exploratoryinput validationnoneno
decision-informingassumption and limitation statementnoneno
decision-informinginput validationnoneno
decision-informingcode execution for arithmeticnoneno
decision-informingsensitivity analysisnoneno
high-stakesassumption and limitation statementnoneno
high-stakesinput validationnoneno
high-stakescode execution for arithmeticnoneno
high-stakessensitivity analysisnoneno
high-stakesindependent reviewneededno
high-stakesexpert reviewneededno
high-stakessecond technique with different assumptionsnoneconditional

Failure modes and limits

Model suitabilityhuman judgement required
Typical model failureFills the conditional probability matrix with plausible guesses that give impossible joint values.
Error sourcesexpert opinion; statistical assumptions
Certainty limitsCertainty drops when judgements come from one group (risk of bias), cannot be reproduced by another group (inconsistency), or are guessed (imprecision).

Evidence and tag basis

Documented. The intelligence-analysis variant depends heavily on analysts' opinion and is hard to reproduce with a different group. (Wikipedia, Cross-impact analysis)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.