CM1-06 Case selection techniques (extreme, deviant, diverse, index, most similar)
Chooses which cases to compare by their position on a variable distribution or on a model prediction, producing a chosen case set with the rule used.
What it does
Chooses which cases to compare by their position on a variable's distribution or on a model's prediction. Artefact: a chosen case set.
When to use it
Use when a user has many possible cases and needs a defensible way to pick a few to study, such as extreme, typical or deviant ones.
When it does not fit
Not established; the 2022 chapter positions these techniques as exploratory, not confirmatory.
Result format
The result is the chosen set of cases, each named with its type (extreme, deviant, diverse, index or most similar) and its position on the variable's distribution or on the model's prediction that led to its selection.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Structure | Applies in some cases | requires=structured |
| Content type | Always applies | requires=tabular |
| Unit of analysis | Always applies | requires=record |
| Variable types | Raises a warning | prefers=interval; prefers=ratio |
| Level of detail available | Always applies | requires=record-level |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Time structure, How the sample was chosen, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | A defined population of cases with data on the variable or model used for selection. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | The notes establish no fit test. The source positions these techniques as exploratory, so a selected case set should not be used to confirm a theory. |
Choosing and ordering
| Kind of technique | Preparation technique, runs in step 5.2 |
|---|---|
| Phase tag | gather |
| Other phase tags | frame |
| Kind of finding | pattern; difference |
| Question answered | what |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to frame |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: deterministic.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | input validation | none | no |
| decision-informing | input validation | none | no |
| decision-informing | code execution for arithmetic | none | no |
| decision-informing | spot check | optional | no |
| high-stakes | input validation | none | no |
| high-stakes | code execution for arithmetic | none | no |
| high-stakes | spot check | optional | no |
| high-stakes | sensitivity analysis | none | no |
| high-stakes | expert review | needed | no |
Failure modes and limits
| Model suitability | model plus code |
|---|---|
| Typical model failure | Selects extreme cases on the outcome and presents them as typical, or applies a selection rule the source does not define. |
| Error sources | input data; source selection; statistical assumptions |
| Certainty limits | Certainty drops because the definitions of the five techniques could not be retrieved, and because selection on the outcome limits any causal reading (risk of bias). |
Related techniques
- should follow MM-01 Exploratory data analysis with plots and distribution checks (EDA (exploratory data analysis); summary statistics; Q-Q plot; scatterplot matrix): the shape of the selection variable should be seen before cases are chosen by position on it
- pairs with CM1-05 Comparative case study (structured focused comparison, small-N comparison): the chosen cases are then compared in depth
Evidence and tag basis
Unconfirmed. Only the list of five techniques was verified; definitions could not be retrieved. (Cambridge, Finding Your Social Science Project, case selection)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.