CM1-03 Mill's method of difference and most similar systems design (MSSD; joint method of agreement and difference)
Looks for the factor present where an outcome occurs and absent in an otherwise similar case where it does not, producing a short list of the remaining differences between matched cases.
What it does
Looks for the factor present where the outcome occurs and absent in an otherwise similar case where it does not. Artefact: the short list of remaining differences between matched cases.
When to use it
Use when a user has a case with an outcome and a close match without it and asks what made the difference.
When it does not fit
The cases still differ on many characteristics. Przeworski and Teune concede that no two countries differ in only two characteristics, so rival hypotheses always remain; cross-national statistics also differ in definition and coverage.
Result format
The result sets the case where the outcome occurs beside an otherwise similar case where it does not, and gives a short list of the remaining differences between the matched cases. It names the factor present where the outcome occurs and absent where it does not.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Content type | Always applies | requires=text; requires=tabular |
| Unit of analysis | Always applies | requires=topic; requires=record |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | At least one case with the outcome, one similar case without it, and detailed background data on both. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Wrong choice when the cases still differ on many characteristics, because rival hypotheses then remain. No two cases differ in only one or two characteristics, and cross-national statistics differ in definition and coverage. |
Choosing and ordering
| Kind of technique | Primary technique, runs in step 5.3 |
|---|---|
| Phase tag | analyse |
| Other phase tags | none |
| Kind of finding | cause; difference |
| Question answered | why; what |
| Strongest statement it supports | show a link between things |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to gather |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | assumption and limitation statement | none | no |
| decision-informing | assumption and limitation statement | none | no |
| decision-informing | alternative explanations check | none | no |
| decision-informing | spot check | optional | no |
| high-stakes | assumption and limitation statement | none | no |
| high-stakes | alternative explanations check | none | no |
| high-stakes | spot check | optional | no |
| high-stakes | second technique with different assumptions | none | conditional |
| high-stakes | expert review | needed | no |
Failure modes and limits
| Model suitability | human judgement required |
|---|---|
| Typical model failure | Treats two cases as matched without checking background differences, and names the single visible difference as the cause. |
| Error sources | input data; source selection; coder or rater judgement |
| Certainty limits | Certainty drops when the matched cases still differ on several factors, when definitions or coverage differ between the cases (inconsistency), or when the pair was chosen after the outcome was known (risk of bias). |
Related techniques
- should follow CM1-06 Case selection techniques (extreme, deviant, diverse, index, most similar): most similar case selection supplies the matched pair
- pairs with CM1-02 Mill's method of agreement and most different systems design (MDSD): the joint method combines agreement and difference, though it is weaker than the direct method of difference
- pairs with TC1-04 Process tracing (diagnostic tests: straw in the wind, hoop, smoking gun, doubly decisive): process tracing tests which remaining difference actually operated
Evidence and tag basis
Documented. As for the method of agreement. The joint method is described as weaker than the direct method of difference. (Britannica, Mill's methods; Wikipedia, Mill's Methods; Przeworski and Teune design summary)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.