TC1-26 Thought experiments
Constructs an imagined case to test a principle and produces a judgement about the case that counts as evidence on the principle only if reasoners agree.
What it does
Constructs an imagined case to test a principle. Artefact: a judgement about the case, which is evidence about the principle only if reasoners agree.
When to use it
Use when a principle cannot be tested on real cases and you want to see whether it gives acceptable answers in a constructed one.
When it does not fit
Details of the case, not the principle, determine the judgement.
Result format
The result describes the imagined case, states the principle it tests and gives the judgement about the case. It notes whether the reasoners agree, since the judgement is evidence about the principle only if they do.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Content type | Always applies | requires=text |
| Unit of analysis | Always applies | requires=claim |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | A principle to test and a constructed case that isolates the feature of interest. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Wrong choice when details of the case, and not the principle, determine the judgement. |
Choosing and ordering
| Kind of technique | Primary technique, runs in step 5.3 |
|---|---|
| Phase tag | analyse |
| Other phase tags | none |
| Kind of finding | contradiction; gap |
| Question answered | what; why |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to analyse |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | assumption and limitation statement | none | no |
| decision-informing | assumption and limitation statement | none | no |
| decision-informing | spot check | optional | no |
| high-stakes | assumption and limitation statement | none | no |
| high-stakes | spot check | optional | no |
| high-stakes | independent review | needed | no |
Failure modes and limits
| Model suitability | human judgement required |
|---|---|
| Typical model failure | Builds a case whose incidental details produce the judgement and counts the model's own reaction as evidence about the principle. |
| Error sources | coder or rater judgement; expert opinion; language model |
| Certainty limits | A judgement from one reasoner is weak evidence. Certainty drops when reasoners disagree or when the result changes with details of the case. |
Related techniques
- pairs with TC1-24 Reflective equilibrium: reflective equilibrium reconciles the case judgement with the principle
Evidence and tag basis
Unconfirmed. The source entry failed to load, so no claim was verified.
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.