startthat.work home

PJ-05 Prediction markets (information markets)

Read the price of a defined contract as a probability for an event, giving a price series and a liquidity-qualified probability as a draft result.

Project forwardEvidence: TestedPrimary technique, runs in step 5.3Strongest statement: predict, without claiming a cause

What it does

Aggregates traders' beliefs into a price read as a probability for a defined event. Artefact: a price series; read the price as a probability only where the market is liquid.

When to use it

Use when a liquid public market already prices the event you care about and you want an aggregated probability.

When it does not fit

Trading in the market is limited, contract wording differs from the question, the horizon is long, or participants can manipulate the price.

Result format

The result is a price series for the defined event, with the price read as a probability. It says whether the market is liquid enough for that reading to hold.

Requirements on the material

PropertyRequirement typeRequirement
StructureApplies in some casesrequires=structured
Content typeAlways appliesrequires=time series
Unit of analysisAlways appliesrequires=topic
Variable typesRaises a warningprefers=counts or fractions
Time structureApplies in some casesrequires=aggregate time series; requires=continuous stream
Level of detail availableAlways appliesrequires=aggregate tables
Appraises source credibilityApplies in some casesno

Not restricted on: How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialA precisely defined contract and enough traders and trading volume for the price to be liquid.
Additional sources needednone
Not for (rules)none
Not for (text)Trading is limited, contract wording differs from the question, the horizon is long, or participants can manipulate the price.

Choosing and ordering

Kind of techniquePrimary technique, runs in step 5.3
Phase taganalyse
Other phase tagsnone
Kind of findingrisk; trend
Question answeredwhat next
Strongest statement it supportspredict, without claiming a cause
Needs as input
Makes as output
Must exist firstNone
Loop targetreturn to analyse

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.

StakesCheckPerson involvedCounts as a different method
exploratoryinput validationnoneno
decision-informinginput validationnoneno
decision-informingspot checkoptionalno
decision-informingsensitivity analysisnoneno
high-stakesinput validationnoneno
high-stakesspot checkoptionalno
high-stakessensitivity analysisnoneno
high-stakessecond technique with different assumptionsnoneconditional
high-stakesassumption and limitation statementnoneno

Failure modes and limits

Model suitabilityhuman judgement required
Typical model failureReads a thin or differently worded market price as a probability for the question.
Error sourcesinput data; expert opinion
Certainty limitsCertainty drops when trading volume is low (imprecision), when contract wording differs from the question (indirectness), or when the price may be manipulated (risk of bias).

Evidence and tag basis

Tested. Market forecasts are fairly accurate across settings and generally beat moderately sophisticated benchmarks. (Wolfers and Zitzewitz (2004), Prediction Markets, Journal of Economic Perspectives; AI Impacts, summary of Good Judgment Project research)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.