startthat.work home

Analysis techniques: what exists, what is used and what has been tested

In short

This report summarises the research behind the catalogue of 236 analysis techniques. It covers which techniques exist, what evidence supports them, how widely they are used and what the research did not cover. It combines two reports on techniques, one for desk research on any topic and one for text and data, with surveys and interviews on technique use and the frameworks that consultancies publish.

The catalogue has 236 techniques in 12 families. Of these, 33 have a published study of their effect or accuracy, 150 are described without a test, 13 are used in practice with no published description found, and 40 have no source that could be opened. The tests are mostly of techniques whose output can be scored, such as forecasting, text classification and checks on model output. None of the 11 techniques for combining sources has a test. Two experiments on analysis of competing hypotheses found no accuracy gain.

The most used techniques in the surveys are SWOT (strengths, weaknesses, opportunities, threats) at nearly 80 per cent of 2,148 UK business school alumni, brainstorming at 91.2 per cent of Ontario government analysts, and interview at 89 per cent of Ukrainian business analysts. The least used are specialised or quantitative, such as Markov chain modelling at 0.8 per cent of analysts. No technique has both a use figure and a measured accuracy figure, so the research cannot say whether widely used techniques give better results.

Practitioners use a handful of tools at a time. Consultancies publish named frameworks under their own brands and report wide use of them, and no independent count was found. The catalogue has gaps: five families were not extended to text and data, and many named methods for data, text and combining sources are missing because no source describing them could be opened.

This page has three levels. The summary above gives the answer. The six themes below group what the research found. The findings under them give the figures, the sources and one caveat each. The last section lists every technique.

What the research shows

1

What techniques exist

The catalogue has 236 techniques in 12 families: eight for desk research on any topic and four added for text and data. The largest family is trace cause and test claims, with 34 techniques, and the smallest is frame the question, with 6. Among the techniques added for text and data, checking extracted results is the largest family, with 29 of 84. Desk evidence alone is enough for 100 of the first 150 techniques.

See findings 1.1 to 1.4

2

What has been tested

Of the 236 techniques, 33 have a published study of their effect or accuracy. The tests are mostly of techniques whose output can be scored, and none of the 11 techniques for combining sources has a test. SWOT, Porter's Five Forces, Cynefin and Wardley maps are widely used, and the research found no controlled study of their effect. Two experiments on analysis of competing hypotheses found no accuracy gain, and the largest reported gains are for forecasting methods. Language model reliability varies by task. The reports conclude that checks built into the procedure matter more than the choice of technique.

See findings 2.1 to 2.8

3

What is used most and least

SWOT is the most used tool in the strategy surveys, at nearly 80 per cent of 2,148 UK business school alumni. Brainstorming is first among Ontario government analysts, at 91.2 per cent, and interview is first among Ukrainian business analysts, at 89 per cent. The least used techniques are specialised or quantitative, such as Markov chain modelling at 0.8 per cent of analysts. Managers in the alumni survey used one to nine tools, and four was the most common number. A consultancy survey shows the average falling from 16.1 tools in 2002 to 7.0 in 2014. Interviews with consultants describe a small toolkit that is adapted to the client and used as support for the work.

See findings 3.1 to 3.5

4

How use differs

Each kind of work has its own leading techniques: SWOT in strategy, brainstorming and consultation in policy, interview and document analysis in business analysis. In policy work, consultants used focus groups, modelling tools and Monte Carlo techniques more than government analysts, and used checklists and brainstorming less. In the alumni survey SWOT was the most used tool and was not among the most valued.

See findings 4.1 to 4.3

5

Where named frameworks come from

Consulting firms publish named frameworks and tools. McKinsey lists classic frameworks by decade, including the 9-box matrix, 7-S, three horizons and the consumer decision journey. BCG published the growth share matrix in 1968. Bain published the Net Promoter Score in 2003, the 30 Elements of Value and The Founder's Mentality, and has run a management tools survey since 1993. PwC's Strategy& publishes Capabilities-Driven Strategy. Management scholars describe consultants as fashion setters who package and spread management ideas. No tool exists for appraising sponsored surveys.

See findings 5.1 to 5.4

6

What the catalogue does not cover

Five of the twelve families were not extended to text and data. Common methods for structured data and for existing reports are missing, as are many named methods for text and for combining sources, because no source describing them could be opened. Four terms from earlier work are not established technique names. No published procedure exists for combining sources that share no identifier.

See findings 6.1 to 6.5

The use figures come from self-reported surveys and interviews, so they show patterns and not exact rates.

The evidence in detail

This section has 29 findings in 6 groups. Each group is one of the 6 themes above, and its findings are numbered from 1.1.

  1. What techniques existfindings 1.1 to 1.4
  2. What has been testedfindings 2.1 to 2.8
  3. What is used most and leastfindings 3.1 to 3.5
  4. How use differsfindings 4.1 to 4.3
  5. Where named frameworks come fromfindings 5.1 to 5.4
  6. What the catalogue does not coverfindings 6.1 to 6.5

Each finding gives what the sources report and ends with one caveat about the source.

1What techniques exist4 findings
1.1The catalogue has 236 techniques in 12 families.

The first eight families are for desk research on any topic: frame the question (6), decompose and map (21), compare (17), trace cause and test claims (34), handle evidence (29), project forward (25), assess actors, options and risks (19) and challenge (15). Four families were added for text and data: prepare and extract (14), measure and model (16), integrate across sources (11) and check extracted results (29). Fifteen text and data techniques fit the first eight families and are counted there.

Caveat The families are the catalogue's own grouping by what a technique does, because no source supplies one.

1.2The text and data techniques work on three kinds of input.

Of the 84 techniques added for text and data, 57 work on unstructured text, 37 on structured data and 24 on existing reports. Thirty-one accept more than one kind of input.

Caveat The counts add to more than 84 because a technique is counted under each input it accepts.

1.3Checking extracted results is the largest family among the text and data techniques.

The family has 29 of the 84 techniques. A model or script that performs an extraction step creates a need to measure its error. Most of the checks come from older statistical practice: agreement coefficients, gold-standard samples, false discovery control and leakage checks. The checks specific to language model output are repeated-run tests, prompt stability scores, specification sweeps, entailment checks, sampling-consistency detection, judge models and calibration.

Caveat The reason given is the report's inference.

1.4Desk evidence alone is enough for 100 of the first 150 techniques.

Of the first 150 techniques, 45 need expert judgement or other inputs that desk sources only partly supply, and 5 need people or data that desk research cannot create. Of the 84 text and data techniques, one can be done by a model alone, 54 need code and 29 need human judgement.

Caveat The fit ratings are mostly inference from the sources.

2What has been tested8 findings
2.1Most techniques have a description but no test of their effect.

Of the 236 techniques, 33 are tested, meaning at least one published study measured the effect or accuracy. 150 are documented, meaning a text, guidance or summary describes them without a test. 13 are practice, meaning they are used with no published description found. 40 are unconfirmed, meaning no source was opened or a source failed to load. A status of documented, practice or unconfirmed means no study was found. It does not mean the technique was tested and found ineffective.

  • Tested
  • Documented
  • Practice
  • Unconfirmed
Frame the question6 0 / 2 / 3 / 1
Decompose and map21 0 / 15 / 2 / 4
Compare17 1 / 12 / 0 / 4
Trace cause and test claims34 5 / 20 / 3 / 6
Handle evidence29 0 / 18 / 1 / 10
Project forward25 5 / 9 / 0 / 11
Assess actors, options and risks19 0 / 17 / 1 / 1
Challenge15 0 / 12 / 2 / 1
Prepare and extract14 5 / 9 / 0 / 0
Measure and model16 7 / 9 / 0 / 0
Integrate across sources11 0 / 10 / 0 / 1
Check extracted results29 10 / 17 / 1 / 1

Bar length is the number of techniques. The numbers after it are tested, documented, practice and unconfirmed, in that order.

Show as a table
Family Techniques Tested Documented Practice Unconfirmed
Frame the question 6 0 2 3 1
Decompose and map 21 0 15 2 4
Compare 17 1 12 0 4
Trace cause and test claims 34 5 20 3 6
Handle evidence 29 0 18 1 10
Project forward 25 5 9 0 11
Assess actors, options and risks 19 0 17 1 1
Challenge 15 0 12 2 1
Prepare and extract 14 5 9 0 0
Measure and model 16 7 9 0 0
Integrate across sources 11 0 10 0 1
Check extracted results 29 10 17 1 1
All families 236 33 150 13 40

Caveat The report on desk research counted studies of effect only. The report on text and data also counted studies of accuracy and reliability.

2.2Tests exist mostly where output can be scored.

Tests exist in 3 of the 8 original families: compare (1), trace cause and test claims (3) and project forward (5). The clearest positive results are for forecasting, where outcomes can be scored. In the text and data techniques, text classification and coding are tested most often because they can be scored against human labels. None of the 11 techniques for combining sources has a test, and no opened study tests a language model on those steps.

Caveat Status was assigned from abstracts and secondary pages for most studies.

2.3Widely used techniques such as SWOT have no controlled study of their effect.

The research found no controlled study of effect for the SWOT matrix, Porter's Five Forces, Cynefin or Wardley maps, although all four are widely used. Critiques of two other techniques are analytical and not experimental. A peer-reviewed commentary finds no evidence of effectiveness for the five whys, and Cox (2008) found that typical risk matrices reliably compare under 10 per cent of random hazard pairs.

Caveat A study may exist that the search did not reach.

2.4Two experiments on analysis of competing hypotheses found no accuracy gain.

In the study of 50 intelligence analysts the mean absolute error was 0.21 for controls and 0.26 for the technique group (p = .14, not significant). A second experiment with graduate-level civilian participants found the technique group slightly less reliable (p = .051, d = 0.20).

Caveat Both experiments come from one research group, and the samples are small.

2.5Forecasting methods show the largest reported accuracy gains.

Good Judgment Project forecasters beat a control group by 60 per cent in year 1 and 78 per cent in year 2, teams were 23 per cent more accurate than individuals, and a training module under one hour improved Brier scores by 6 to 11 per cent. A separate review found that prediction markets generally beat moderately sophisticated benchmarks.

Caveat The Good Judgment figures come from a blog summary.

2.6Language model reliability depends on the task.

Across 13 million labels from 18 models, about 31 per cent of tested hypotheses reached wrong conclusions. On RADAR, a table-question benchmark, one model fell from 83 per cent on clean tables to 32 per cent on logically inconsistent ones.

Caveat The studies date from 2023 to 2025 and may not describe current models.

2.7Arithmetic checks find inconsistencies in about half of the articles that report results in a standard format.

Across 30,717 articles, statcheck found at least one inconsistency in 49.6 per cent of those that reported results in the American Psychological Association style. A pilot of the ROBIS instrument for appraising reviews gave 65 per cent agreement between two reviewers.

Caveat The ROBIS figure is from one pilot.

2.8Checks built into the procedure matter more than the choice of technique.

The reports conclude that the quality of a desk analysis depends mainly on checks built into the procedure, because measured effects exist for so few techniques. The techniques with such a check have an outcome that can be resolved, such as scoring forecasts or comparing an extrapolation with a naive no-change forecast. A researcher who picks a text or table technique also needs a validation technique.

Caveat This is the reports' inference from the pattern of tests.

3What is used most and least5 findings
3.1SWOT, brainstorming and interviews lead in their kinds of work.

SWOT analysis is the most used tool in the strategy surveys (nearly 80 per cent of 2,148 UK business school alumni), brainstorming is first among Ontario government analysts (91.2 per cent) and interview is first among Ukrainian business analysts (89 per cent). The review of 27 strategy studies reports that no single tool was most used across four global studies, and that benchmarking and SWOT each appeared in three.

Caveat The figures are self-reported.

3.2The least used techniques are specialised or quantitative.

Among analysts, Markov chain modelling was used by 0.8 per cent and preference scaling by 7.0 per cent, and free-form gaming or policy exercises by 6.2 per cent. In the alumni survey Bowman's strategy clock and corporate parenting matrices were hardly used.

Caveat The policy figures come from one survey with inconsistent tables.

3.3Practitioners use a handful of tools at a time.

In the alumni survey managers used one to nine tools, with four the most common number. Interviewees in the German consultant study said they mainly use five to ten tools. A consultancy survey gives an average of 7.0 tools in 2014.

Caveat The German figure is what consultants said they use, not observed practice.

3.4Consultancies report using fewer tools over time.

The Bain figures give an average of 16.1 tools in 2002, 7.4 in 2012 and 7.0 in 2014, and rankings in which benchmarking was first in 2008 and 2010 and fourth in 2013.

Caveat The figures come from a consultancy survey.

3.5Consultants use a small, adapted toolkit as support.

In the German study of 12 strategy consultants, interviewees mainly use five to ten tools, success was linked to adapting textbook tools to the client, and seven of 12 said tools should not define the core of the work. The Finnish study of 17 senior people lists methods and tasks without frequencies and reports generic approaches and over-reliance as challenges.

Caveat The evidence is two master's theses with no per-tool counts.

4How use differs3 findings
4.1The leading techniques differ between kinds of work.

The strategy surveys put SWOT and benchmarking first, the policy survey puts brainstorming and consultation first, the business analysis survey puts interview and document analysis first, and one intelligence study reports that 1 in 3 analysts never used the structured techniques. Some techniques appear on several lists (cost-benefit analysis in the strategy and policy surveys, scenario analysis in the alumni and policy surveys), so the kinds of work differ in their leading techniques, and not in whole sets of techniques.

Caveat Each survey asked about a different list of tools, so the results cannot be compared directly.

4.2Consultants use more modelling and focus groups than government analysts.

Consultants used focus groups (57.3 against 37.8 per cent), sophisticated modelling tools (26.7 against 11.2) and Monte Carlo techniques (10.4 against 1.5) more than government analysts. They used checklists (33.3 against 60.1) and brainstorming (69.7 against 82.5) less. The survey is about policy consulting.

Caveat The consultant response rate was 13 per cent.

4.3The most used tool, SWOT, was not among the most valued.

The alumni survey reports that SWOT is the most used analysis tool and is not among the most valued, scenario planning is among the most valued for strategy choice and not among the most used, and Porter's five forces and PESTLE are rated most valuable for analysis. In the intelligence study, use was related to analytic training and perceived value.

Caveat This rests on one survey and one intelligence study.

5Where named frameworks come from4 findings
5.1Consultancies publish named frameworks and tools under their own brands.

McKinsey's own list of classic frameworks gives, by decade: the 9-box matrix, 7-S framework and business system (1970s); the industry cost curve and SCP framework (1980s); the strategic control map and three horizons (1990s); the portfolio of initiatives (2000s); and the consumer decision journey (2010s). BCG says it created the growth share matrix in 1968 and also names time-based competition. Bain says Fred Reichheld created the Net Promoter Score in 2003, and Bain also publishes the 30 Elements of Value, The Founder's Mentality (a 2016 book by Chris Zook and James Allen) and a yearly guide to 25 management tools from a research project that began in 1993. PwC's Strategy& publishes Capabilities-Driven Strategy and a Strategy Profiler tool.

Caveat These are the firms' own statements of what they published.

5.2Firms report wide use of their own frameworks.

BCG says that in the late 1970s and early 1980s about half of all Fortune 500 companies used the growth share matrix or approaches based on it. Fortune reports that at least two-thirds of the Fortune 1000 use the Net Promoter Score. No independent count was found for either.

Caveat The BCG figure is the firm's claim.

5.3Scholars describe consultants as setters of management fashions.

Research on management fashion names consulting firms, management gurus, business media and business schools as fashion setters. It describes consultants as diffusion agents who popularise management ideas and turn abstract concepts into commercial products.

Caveat This comes from summaries of the research, not the papers.

5.4No tool exists for appraising sponsored surveys.

The research found no study or tool for appraising sponsored vendor or consultancy surveys. AACODS, the one checklist built for grey literature, has no item on funding. AMSTAR 2 asks about funding, and both it and ROBIS are written for health reviews. Health research on drug and device studies found that industry-sponsored studies more often reported favourable results (risk ratio 1.27) and favourable conclusions (1.34). A 2024 commentary argues that funding alone should not lower a risk of bias rating.

Caveat The health findings apply to surveys by inference.

6What the catalogue does not cover5 findings
6.1Five of the twelve families were not extended to text and data.

The report on text and data did not search frame the question, decompose and map, project forward, assess actors, options and risks, or challenge. Data-driven forecasting, for example, depends only on the earlier trend extrapolation and decomposition rows.

Caveat This is a limit of the research and not evidence that text and data versions do not exist.

6.2Common methods for structured data and existing reports are missing.

Structured data alone has 14 techniques. Association, regression, cohort, change point and cross-tabulation analysis are absent. Existing reports alone have 12 techniques, and none checks percentages and totals or re-analyses published tables. Techniques that join text to tables, such as entity resolution, truth discovery, copy detection and the five mixed methods designs, have no test.

Caveat The methods are missing because no source describing them was opened, not because they lack one.

6.3Many named methods were left out because no opened source described them.

The structured data notes named duplicate and key checks, cross-tabulation, cohort analysis, regression, classification trees, change point detection, Pareto analysis, geospatial analysis, control charts, interrupted time series, propensity score matching and Granger causality tests, among others. The text notes named TF-IDF (term frequency and inverse document frequency), embedding-based semantic search, argument mining, text scaling and claim detection. The notes for combining sources named triangulation types, framework synthesis, realist synthesis, Dempster-Shafer evidence combination and Bayesian evidence synthesis. The notes for existing reports named percentage and total reconciliation, replication checks, p-curve analysis and the AXIS, RoB 2 and AGREE II appraisal instruments. The report on desk research also lists synthetic control, matching, Kepner-Tregoe analysis, concept mapping, Pareto analysis, attack trees, force field analysis and the decision matrix as not researched.

Caveat A technique is catalogued only if an opened source describes or tests it.

6.4Four terms from earlier work are not established technique names.

Trajectory analysis, absence analysis, false commensurability and contrast with a structural opposite have no defining source. The technique names attributed to Heuer and Pherson are unconfirmed.

Caveat The reports treat these as leads and not as techniques.

6.5No published procedure exists for combining sources that share no identifier.

The suggested order has five steps: define the unit, code every source to one written reference structure, resolve entities by attribute matching, record the provenance of each item, and only then compare, fuse or weight. No published procedure was found for this order, and no source gives accuracy for attribute matching without identifiers.

Caveat The order is the report's own synthesis.

?What the research did not find5 items

The research searched for each item below and found nothing. It searched web pages and summaries of them, so a study or survey may exist that the search did not reach. Firms other than McKinsey, BCG, Bain, PwC, Deloitte and Accenture were not searched.

  • A ranking of techniques by use across all desk research. The surveys cover strategy, policy, business analysis and intelligence work, each with its own list of tools.
  • Use figures for data analysis, text analysis and evidence appraisal outside health.
  • An independent count of the methods consultancies use.
  • Named frameworks from Deloitte and Accenture.
  • A technique with both a use figure and a measured accuracy figure, so the page cannot say whether popular techniques give better results.

The techniques

The catalogue lists 236 techniques. Each row below gives the identifier, the name and the evidence status. The description, fit test and evidence for each technique are in the catalogue. Two techniques, AO-19 and PE-14, are in the catalogue and in neither report, which counted 234. Both are documented.

  • Tested
  • Documented
  • Practice
  • Unconfirmed
Frame the question6
IDTechniqueEvidence status
FQ-01PICO familyDocumented
FQ-02Decision-first framingUnconfirmed
FQ-03Definitional analysisDocumented
FQ-04Reframing and issue redefinitionPractice
FQ-05Multiple mental modelsPractice
FQ-06SMART checkPractice
Decompose and map21
IDTechniqueEvidence status
DM-01MECE decomposition and issue treePractice
DM-02Hypothesis treeUnconfirmed
DM-03Morphological analysisDocumented
DM-04Value chain analysisDocumented
DM-05Taxonomy buildingDocumented
DM-06Affinity diagramDocumented
DM-07Lens applicationUnconfirmed
DM-08Two by two matrixUnconfirmed
DM-09PESTLEDocumented
DM-10SWOTDocumented
DM-11Porter's Five ForcesDocumented
DM-12Cynefin frameworkDocumented
DM-13System archetypesDocumented
DM-14Leverage pointsDocumented
DM-15Wardley mappingPractice
DM-16Levels-of-analysis modelsDocumented
DM-17Process mappingUnconfirmed
DM-18Design structure matrixDocumented
DM-19Causal loop diagramDocumented
DM-20Stock and flow modellingDocumented
DM-21Social network analysisDocumented
Compare17
IDTechniqueEvidence status
CM1-01Historical analogy and parallel analysisDocumented
CM1-02Mill's method of agreement and most different systems designDocumented
CM1-03Mill's method of difference and most similar systems designDocumented
CM1-04Concomitant variation and method of residuesDocumented
CM1-05Comparative case studyDocumented
CM1-06Case selection techniquesUnconfirmed
CM1-07Qualitative comparative analysisDocumented
CM1-08Cross-case synthesisUnconfirmed
CM1-09Contrast with a structural oppositeUnconfirmed
CM1-10Cross-sectional and longitudinal comparisonDocumented
CM1-11Before and after comparisonUnconfirmed
CM1-12Benchmarking and peer comparisonDocumented
CM1-13Forecaster and source comparisonTested
CM1-14Absence analysisDocumented
CM1-15Gap analysisDocumented
CM1-16Maturity model and capability assessmentDocumented
CM-01Sponsor versus non-sponsor comparisonDocumented
Trace cause and test claims34
IDTechniqueEvidence status
TC1-01Five whysPractice
TC1-02Fishbone diagramDocumented
TC1-03Fault tree analysisDocumented
TC1-04Process tracingDocumented
TC1-05Counterfactual analysisDocumented
TC1-06Difference-in-differencesDocumented
TC1-07Quasi-experimental designsUnconfirmed
TC1-08Confounder identification with directed acyclic graphsUnconfirmed
TC1-09Logic model and theory of changeDocumented
TC1-10Contribution analysisDocumented
TC1-11Realist evaluationDocumented
TC1-12Process evaluation and impact evaluationDocumented
TC1-13Analysis of competing hypothesesTested
TC1-14Coherence correction and averaging of probability judgementsTested
TC1-15Bayesian updatingUnconfirmed
TC1-16Disconfirmation searchDocumented
TC1-17Sensitivity analysisDocumented
TC1-18Internal consistency checkUnconfirmed
TC1-19Toulmin argument analysisDocumented
TC1-20Argument mappingUnconfirmed
TC1-21Fallacy analysisDocumented
TC1-22Abduction and inference to the best explanationDocumented
TC1-23Ladder of inferencePractice
TC1-24Reflective equilibriumDocumented
TC1-25First principles reasoningPractice
TC1-26Thought experimentsUnconfirmed
TC1-27Fermi estimation and order-of-magnitude checksTested
TC1-28Dimensional and unit checksDocumented
TC-01GRIM testDocumented
TC-02SPRITEDocumented
TC-03StatcheckTested
TC-04Claim and citation verification against source textTested
TC-05Subgroup and Simpson's paradox checkDocumented
TC-06Ecological fallacy checkDocumented
Handle evidence29
IDTechniqueEvidence status
HE1-01Admiralty systemDocumented
HE1-02Quality of information checkDocumented
HE1-03CRAAP testDocumented
HE1-04SIFTPractice
HE1-05Lateral readingUnconfirmed
HE1-06TriangulationUnconfirmed
HE1-07Evidence hierarchies and certainty gradingDocumented
HE1-08Systematic reviewDocumented
HE1-09Rapid reviewDocumented
HE1-10Scoping reviewUnconfirmed
HE1-11Meta-analysisDocumented
HE1-12Narrative, thematic and qualitative evidence synthesisDocumented
HE1-13Reflexive thematic analysisDocumented
HE1-14Framework analysisDocumented
HE1-15Content analysisUnconfirmed
HE1-16Discourse analysisUnconfirmed
HE1-17Grounded theory codingUnconfirmed
HE1-18Document and policy analysisUnconfirmed
HE1-19Bibliometric analysisUnconfirmed
HE1-20Trend and time-series decompositionUnconfirmed
HE1-21Distribution, outlier and base-rate checksUnconfirmed
HE-01Grey literature appraisal checklistDocumented
HE-02Critical appraisal checklist by study designDocumented
HE-03Appraisal of a systematic reviewDocumented
HE-04Survey disclosure completeness checkDocumented
HE-05Margin of error and credibility interval checkDocumented
HE-06Opt-in panel respondent quality checkDocumented
HE-07Overview of reviewsDocumented
HE-08Living systematic reviewDocumented
Project forward25
IDTechniqueEvidence status
PJ-01Trend extrapolation and time-series forecastingUnconfirmed
PJ-02Reference class forecastingTested
PJ-03Forecasting tournaments and superforecasting practiceTested
PJ-04Delphi techniqueDocumented
PJ-05Prediction marketsTested
PJ-06Structured expert elicitationUnconfirmed
PJ-07Technology S-curve analysisUnconfirmed
PJ-08Bass diffusion modelDocumented
PJ-09Hype cycleTested
PJ-10Trajectory analysisUnconfirmed
PJ-11Agent-based modelling and Monte Carlo simulationUnconfirmed
PJ-12Scenario planningTested
PJ-13Scenario matrixDocumented
PJ-14Alternative futures archetypesUnconfirmed
PJ-15Futures wheelUnconfirmed
PJ-16Causal layered analysisDocumented
PJ-17Cross-impact analysisDocumented
PJ-18BackcastingDocumented
PJ-19Policy stress-testingDocumented
PJ-20Wargaming and simulation exercisesUnconfirmed
PJ-21Robust decision making, adaptive pathways and info-gap analysisUnconfirmed
PJ-22Technology roadmappingUnconfirmed
PJ-23Horizon scanningDocumented
PJ-24Weak signal detectionUnconfirmed
PJ-25Leading indicators and signpostsDocumented
Assess actors, options and risks19
IDTechniqueEvidence status
AO-01Stakeholder analysisDocumented
AO-02Power-interest gridDocumented
AO-03Stakeholder salience modelDocumented
AO-04Political economy analysisDocumented
AO-05Customer journey mappingDocumented
AO-06PersonasDocumented
AO-07Jobs to be donePractice
AO-08Eightfold pathDocumented
AO-09Longlist-to-shortlist options analysisDocumented
AO-10Cost-benefit analysisDocumented
AO-11Cost-effectiveness and cost-utility analysisDocumented
AO-12Multi-criteria decision analysisDocumented
AO-13Decision tree and expected valueDocumented
AO-14Real options analysisUnconfirmed
AO-15Risk matrixDocumented
AO-16Failure mode and effects analysisDocumented
AO-17Bow-tie analysisDocumented
AO-18Threat modellingDocumented
AO-19Ranking uncertainty analysisDocumented
Challenge15
IDTechniqueEvidence status
CH-01Key assumptions checkDocumented
CH-02Strategic assumption surfacing and testingDocumented
CH-03Devil's advocacyDocumented
CH-04Team A and Team BDocumented
CH-05Red team analysisDocumented
CH-06High-impact and low-probability analysisDocumented
CH-07What if analysisDocumented
CH-08PremortemPractice
CH-09Structured self-critiqueDocumented
CH-10Rapoport's rulesDocumented
CH-11SteelmanningPractice
CH-12Socratic questioningDocumented
CH-13Cognitive bias checks and debiasingUnconfirmed
CH-14Structured brainstormingDocumented
CH-15Group process safeguardsDocumented
Prepare and extract14
IDTechniqueEvidence status
PE-01Corpus construction and samplingDocumented
PE-02Preprocessing sensitivity testingTested
PE-03Near-duplicate detectionDocumented
PE-04Data profiling and rule-based validationDocumented
PE-05Missing data diagnosis and treatmentDocumented
PE-06Keyword in contextDocumented
PE-07Collocation and n-gram extractionDocumented
PE-08Named entity recognitionDocumented
PE-09Relation and event extractionTested
PE-10Schema-based structured extraction with an LLMTested
PE-11Knowledge graph construction from textDocumented
PE-12Model summarisation of documentsTested
PE-13Retrieval-augmented analysisTested
PE-14De-identification and re-identification risk assessmentDocumented
Measure and model16
IDTechniqueEvidence status
MM-01Exploratory data analysis with plots and distribution checksDocumented
MM-02Algorithmic outlier and anomaly detectionDocumented
MM-03Clustering of recordsDocumented
MM-04Dimensionality reductionDocumented
MM-05Survival and duration analysisDocumented
MM-06Survey weighting and subgroup estimationTested
MM-07Supervised classification and dictionary methodsTested
MM-08LLM coding and zero-shot classification of textTested
MM-09Sentiment analysisDocumented
MM-10Stance detectionTested
MM-11Framing analysis at scaleTested
MM-12Probabilistic topic modellingTested
MM-13Embedding-based topic modellingDocumented
MM-14Document clusteringDocumented
MM-15Computer-assisted theme buildingDocumented
MM-16Word embeddings for concept measurement and semantic changeTested
Integrate across sources11
IDTechniqueEvidence status
IS-01Entity resolution and linkingDocumented
IS-02Schema matching and ontology matchingDocumented
IS-03Common data model harmonisationDocumented
IS-04Data fusion with conflict resolutionDocumented
IS-05Truth discovery with source reliability weightingDocumented
IS-06Source dependence and copy detectionDocumented
IS-07Convergent comparison of qualitative and quantitative resultsDocumented
IS-08Joint displayDocumented
IS-09Quantitising and qualitisingUnconfirmed
IS-10Explanatory sequential integrationDocumented
IS-11Exploratory sequential integrationDocumented
Check extracted results29
IDTechniqueEvidence status
CK-01Inter-coder agreement statisticsDocumented
CK-02Model versus human coding agreement protocolTested
CK-03Bootstrap confidence intervals for agreement and accuracyDocumented
CK-04Gold-standard validation sampleDocumented
CK-05Out-of-sample validationDocumented
CK-06Bias correction of model labels with a human-coded subsampleTested
CK-07Split-sample design for measures discovered from textDocumented
CK-08Semantic, predictive and convergent validity checksDocumented
CK-09Cluster validation indexesDocumented
CK-10Repeated-run consistency testTested
CK-11Prompt stability scoringTested
CK-12Specification sweep across models and promptsTested
CK-13Multiple-testing correctionDocumented
CK-14Data leakage auditDocumented
CK-15Sample representativeness auditDocumented
CK-16Entailment-based consistency checkTested
CK-17Source-span and quotation check by codeDocumented
CK-18Numeric and attribution recomputation by codeUnconfirmed
CK-19Atomic claim decomposition and scoringTested
CK-20Sampling-consistency hallucination detectionTested
CK-21Reference-free grounding metrics for retrieval-augmented answersDocumented
CK-22LLM-as-judge with human calibrationTested
CK-23Confidence calibration checkTested
CK-24Review sampling designDocumented
CK-25Error taxonomy and error analysisDocumented
CK-26Independent human review of model outputDocumented
CK-27Adversarial testing of an extraction pipelineDocumented
CK-28Provenance tracking and audit loggingDocumented
CK-29Scripted reproducible workflowPractice