startthat.work home

PJ-11 Agent-based modelling and Monte Carlo simulation (microsimulation)

Simulate agents or random draws many times to give an outcome distribution with tail probabilities and the inputs that account for the spread, as a draft result.

Project forwardEvidence: UnconfirmedPrimary technique, runs in step 5.3Strongest statement: predict, without claiming a cause

What it does

Simulates individual agents or random draws many times. Artefact: an outcome distribution; read tail probabilities and which inputs account for the spread.

When to use it

Use when the outcome depends on several uncertain inputs or on the behaviour of many actors and you need a distribution, not a single estimate.

When it does not fit

Input distributions are invented or the model was never validated against a historical outcome.

Result format

The result is an outcome distribution from the repeated simulation runs. It gives the tail probabilities and names the inputs that account for most of the spread.

Requirements on the material

PropertyRequirement typeRequirement
Content typeAlways appliesrequires=tabular
Unit of analysisAlways appliesrequires=variable; requires=topic
Variable typesRaises a warningprefers=ratio; prefers=counts or fractions
Level of detail availableAlways appliesrequires=aggregate tables
Appraises source credibilityApplies in some casesno

Not restricted on: Structure, Time structure, How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialBehaviour rules or distributions for each input variable and calibration data for them.
Additional sources neededreference structure
Not for (rules)none
Not for (text)Input distributions are invented or the model was never validated against a historical outcome.

Choosing and ordering

Kind of techniquePrimary technique, runs in step 5.3
Phase taganalyse
Other phase tagsnone
Kind of findingrisk; pattern
Question answeredhow much; what next
Strongest statement it supportspredict, without claiming a cause
Needs as input
Makes as output
Must exist firstNone
Loop targetreturn to gather

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: repeatable with fixed settings.

StakesCheckPerson involvedCounts as a different method
exploratoryinput validationnoneno
decision-informinginput validationnoneno
decision-informingheld-out validationnoneno
decision-informingsensitivity analysisnoneno
high-stakesinput validationnoneno
high-stakesheld-out validationnoneno
high-stakessensitivity analysisnoneno
high-stakesleakage checknoneno
high-stakessecond technique with different assumptionsnoneconditional
high-stakesassumption and limitation statementnoneno

Failure modes and limits

Model suitabilitymodel plus code
Typical model failureChooses input distributions without a stated basis and reports tail probabilities as if the model had been validated.
Error sourcesinput data; statistical assumptions; expert opinion
Certainty limitsCertainty drops when inputs are invented (risk of bias), when the model was not validated against history (indirectness), or when few runs are made (imprecision).

Evidence and tag basis

Unconfirmed. No source was opened.

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.