startthat.work home

CM1-02 Mill's method of agreement and most different systems design (MDSD)

Looks across dissimilar cases that share an outcome for the factor present in all of them, producing a table of candidate factors by case in which the common factor is the candidate cause.

CompareEvidence: DocumentedPrimary technique, runs in step 5.3Strongest statement: show a link between things

What it does

Looks for the factor present in every case that shows the outcome, across dissimilar contexts. Artefact: a table of candidate factors by case; the factor common to all is the candidate cause.

When to use it

Use when a user has several different cases with the same outcome and asks what they have in common.

When it does not fit

The candidate list is incomplete, the shared factor differs in kind from case to case, or the result is read as confirmation. Agreement only rules out factors that were absent when the effect occurred.

Result format

The result is a table of candidate factors by case, for cases that show the outcome in dissimilar contexts, with each cell showing whether the factor is present. It names the factor common to all cases as the candidate cause.

Requirements on the material

PropertyRequirement typeRequirement
Content typeAlways appliesrequires=text; requires=tabular
Unit of analysisAlways appliesrequires=topic; requires=record
Appraises source credibilityApplies in some casesno

Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialAt least two cases with the same outcome, deliberately dissimilar in context, and an explicit list of candidate factors with comparable data.
Additional sources needednone
Not for (rules)none
Not for (text)Wrong choice when the candidate list is incomplete, when the shared factor differs in kind from case to case, or when agreement is read as confirmation. Agreement only rules out factors that were absent when the effect occurred.

Choosing and ordering

Kind of techniquePrimary technique, runs in step 5.3
Phase taganalyse
Other phase tagsnone
Kind of findingcause; pattern
Question answeredwhy; what
Strongest statement it supportsshow a link between things
Needs as input
Makes as output
Must exist firstNone
Loop targetreturn to gather

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.

StakesCheckPerson involvedCounts as a different method
exploratoryassumption and limitation statementnoneno
decision-informingassumption and limitation statementnoneno
decision-informingalternative explanations checknoneno
decision-informingspot checkoptionalno
high-stakesassumption and limitation statementnoneno
high-stakesalternative explanations checknoneno
high-stakesspot checkoptionalno
high-stakessecond technique with different assumptionsnoneconditional
high-stakesexpert reviewneededno

Failure modes and limits

Model suitabilityhuman judgement required
Typical model failureDeclares the shared factor the cause without checking that the candidate list is complete, and treats factors that differ in kind as the same factor.
Error sourcesinput data; source selection; coder or rater judgement
Certainty limitsCertainty drops when the candidate list may be incomplete or the outcome has several causes (indirectness), and when cases were selected on the outcome (risk of bias).

Evidence and tag basis

Documented. Described in an encyclopaedia entry and a design summary of Przeworski and Teune (1970). Plurality-of-causes criticisms were not in a retrieved source. (Britannica, Mill's methods; Wikipedia, Mill's Methods; Przeworski and Teune design summary)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.