startthat.work home

HE1-14 Framework analysis (Framework Method)

Sorts coded qualitative data into a case by theme matrix and produces the matrix for reading across cases and themes.

Handle evidenceEvidence: DocumentedPrimary technique, runs in step 5.3Strongest statement: describe

What it does

Sorts coded data into a case by theme matrix. Artefact: a matrix; read across rows for cases, down columns for themes, and look for empty cells.

When to use it

Use when you have many cases such as interviews and need to compare themes across them with a visible audit trail.

When it does not fit

The data are heterogeneous, or the spreadsheet look tempts counting of responses. Documents may call for discourse or narrative analysis instead.

Result format

The result is a matrix with one row per case and one column per theme, each cell holding a short summary of the coded data for that case and theme. It notes the empty cells and states what reading across the rows and down the columns shows.

Requirements on the material

PropertyRequirement typeRequirement
Content typeAlways appliesrequires=text
Unit of analysisAlways appliesrequires=document; requires=record
Amount of materialApplies in some casesrequires=readable by a person
Appraises source credibilityApplies in some casesno

Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialsimilar topics across cases; an experienced qualitative lead
Additional sources needednone
Not for (rules)none
Not for (text)The data are heterogeneous, or the spreadsheet layout tempts counting of responses. Documents may call for discourse or narrative analysis instead.

Choosing and ordering

Kind of techniquePrimary technique, runs in step 5.3
Phase taganalyse
Other phase tagsnone
Kind of findingpattern; difference; gap
Question answeredwhat; why
Strongest statement it supportsdescribe
Needs as input
Makes as output
Must exist firstNone
Loop targetproceed with limit stated

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.

StakesCheckPerson involvedCounts as a different method
exploratoryspot checkoptionalno
decision-informingspot checkoptionalno
decision-informinghuman review sampleneeded; left out below high stakesno
high-stakesspot checkoptionalno
high-stakeshuman review sampleneededno
high-stakesindependent reviewneededno
high-stakessecond technique with different assumptionsnoneconditional

Failure modes and limits

Model suitabilityhuman judgement required
Typical model failureFills matrix cells with summaries that drift from the data, and reports counts of cells as response frequencies.
Error sourcesinput data; coder or rater judgement
Certainty limitsCertainty drops with heterogeneous cases, a single coder, and summaries that no longer trace to the data.

Evidence and tag basis

Documented. Seven stages are documented; it is time consuming and needs an experienced qualitative lead. (Gale et al. (2013), the Framework Method, BMC Medical Research Methodology)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.