startthat.work home

PJ-20 Wargaming and simulation exercises (policy gaming, tabletop exercise)

Have players take the roles of competing actors in a structured scenario, giving a record of decisions and surprises as a draft result.

Project forwardEvidence: UnconfirmedPrimary technique, runs in step 5.3Strongest statement: describe

What it does

Has human players take the roles of competing actors in a structured scenario. Artefact: a record of decisions and surprises; read unexpected moves and points where the plan failed, not the outcome as a prediction.

When to use it

Use when you need to see how real people in the roles of competing actors respond to a plan and where it breaks down.

When it does not fit

Players do not represent the actors, one outcome is reported as what will happen, or adjudicators steer the result.

Result format

The result is a record of the scenario, the roles played and the decisions each player made, in the order they were made. It lists the unexpected moves and the points where the plan failed, and does not present the outcome as a prediction.

Requirements on the material

PropertyRequirement typeRequirement
Unit of analysisAlways appliesrequires=topic
Appraises source credibilityApplies in some casesno

Not restricted on: Structure, Content type, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialA scenario, role descriptions, adjudication rules and players who know the actors they represent.
Additional sources neededreference structure
Not for (rules)none
Not for (text)Players do not represent the actors, one outcome is reported as what will happen, or adjudicators steer the result.

Choosing and ordering

Kind of techniquePrimary technique, runs in step 5.3
Phase taganalyse
Other phase tagsvalidate
Kind of findingrisk; contradiction
Question answeredwhat next; who
Strongest statement it supportsdescribe
Needs as input
Makes as output
Must exist firstNone
Loop targetreturn to gather

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.

StakesCheckPerson involvedCounts as a different method
exploratoryassumption and limitation statementnoneno
exploratoryprovenance lognoneno
decision-informingassumption and limitation statementnoneno
decision-informingprovenance lognoneno
decision-informingindependent reviewneeded; left out below high stakesno
decision-informingexpert reviewneeded; left out below high stakesno
high-stakesassumption and limitation statementnoneno
high-stakesprovenance lognoneno
high-stakesindependent reviewneededno
high-stakesexpert reviewneededno
high-stakessecond technique with different assumptionsnoneconditional
high-stakessource-span citationnoneno

Failure modes and limits

Model suitabilityhuman judgement required
Typical model failurePlays all actors itself and reports one simulated outcome as a prediction.
Error sourcesexpert opinion; source selection
Certainty limitsOutputs are insight, not prediction; confidence drops when players do not represent the actors (indirectness) or adjudication steers the result (risk of bias).

Evidence and tag basis

Unconfirmed. No source on predictive accuracy was found; the usual position, not verified here, is that wargames generate insight and do not predict.

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.