CM1-04 Concomitant variation and method of residues (Mill)
Infers a link when a factor varies with the effect (concomitant variation) or attributes the unexplained part of an effect to the remaining circumstances (residues), producing a co-movement pattern or a residual.
What it does
Concomitant variation infers a link when a factor's variation tracks variation in the effect; residues attributes the unexplained part of an effect to the remaining circumstances. Artefact: a co-movement pattern or a residual.
When to use it
Use when a user has an effect that varies in degree and asks whether it moves with a candidate factor, or what is left once known causes are removed.
When it does not fit
Covariation is read as proof, which cannot rule out alternative explanations, or the causes of the other parts of the effect are not established.
Result format
The result is either a co-movement pattern showing how variation in a factor tracks variation in the effect, or a residual showing the part of the effect left unexplained once known factors are removed and the remaining circumstances it is attributed to. It states which of the two it gives.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Content type | Always applies | requires=tabular; requires=time series |
| Unit of analysis | Always applies | requires=variable; requires=record |
| Variable types | Raises a warning | prefers=interval; prefers=ratio; prefers=ordinal |
| Level of detail available | Always applies | requires=aggregate tables |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Time structure, How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | At least five paired observations of the factor and the effect; for residues, known causes for the other parts of the effect. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Wrong choice when covariation is read as proof, because it cannot rule out alternative explanations, or when the causes of the other parts of the effect are not established. |
Choosing and ordering
| Kind of technique | Primary technique, runs in step 5.3 |
|---|---|
| Phase tag | analyse |
| Other phase tags | none |
| Kind of finding | pattern; cause |
| Question answered | why; how much |
| Strongest statement it supports | show a link between things |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to analyse |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: deterministic.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | input validation | none | no |
| decision-informing | input validation | none | no |
| decision-informing | code execution for arithmetic | none | no |
| decision-informing | alternative explanations check | none | no |
| high-stakes | input validation | none | no |
| high-stakes | code execution for arithmetic | none | no |
| high-stakes | alternative explanations check | none | no |
| high-stakes | sensitivity analysis | none | no |
| high-stakes | second technique with different assumptions | none | conditional |
Failure modes and limits
| Model suitability | model plus code |
|---|---|
| Typical model failure | Reports a correlation computed in text with arithmetic slips, and reads co-movement as a cause. |
| Error sources | input data; statistical assumptions; data transformation |
| Certainty limits | Certainty drops when a third factor could drive both series (indirectness), when there are few observations (imprecision), or when the other causes in a residues argument are not established. |
Related techniques
- pairs with TC-05 Subgroup and Simpson's paradox check (stratified reanalysis; confounding check; collapsibility check): a subgroup check tests whether the co-movement reverses within groups
- should follow MM-01 Exploratory data analysis with plots and distribution checks (EDA (exploratory data analysis); summary statistics; Q-Q plot; scatterplot matrix): plots and distribution checks show whether the relationship is plausible before it is read
Evidence and tag basis
Documented. Described in the same sources as the other Mill techniques. (Britannica, Mill's methods; Wikipedia, Mill's Methods)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.