TC1-21 Fallacy analysis (informal fallacies, pragma-dialectical rules)
Compares an argument with a list of known defective patterns and produces a list of suspected defects, each read as a prompt for missing support.
What it does
Compares an argument with a list or rule set of known defective patterns. Artefact: a list of suspected defects; read each as a prompt for missing support, not as proof of falsity.
When to use it
Use when an argument sounds convincing and you want to check whether it relies on a recognised faulty pattern of reasoning.
When it does not fit
A label is applied to a plausible argument because it resembles a pattern.
Result format
The result is a list of suspected defects, each giving the passage of the argument, the named fallacy or rule broken, and the support that appears to be missing. It states that each entry is a prompt for missing support and not proof that the claim is false.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Content type | Always applies | requires=text |
| Unit of analysis | Always applies | requires=claim |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | An argument stated in explicit form, with premises and conclusion separated. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Misused when a label is applied to a plausible argument because it resembles a pattern. A label is a prompt for missing support and not proof that the conclusion is false. |
Choosing and ordering
| Kind of technique | Primary technique, runs in step 5.3 |
|---|---|
| Phase tag | analyse |
| Other phase tags | validate |
| Kind of finding | gap; contradiction |
| Question answered | what; why |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | proceed with limit stated |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | spot check | optional | no |
| decision-informing | spot check | optional | no |
| decision-informing | source-span citation | none | no |
| high-stakes | spot check | optional | no |
| high-stakes | source-span citation | none | no |
| high-stakes | independent review | needed | no |
Failure modes and limits
| Model suitability | model alone |
|---|---|
| Typical model failure | Applies a fallacy name to a sound argument because it resembles the pattern and presents the label as a refutation. |
| Error sources | input data; language model; coder or rater judgement |
| Certainty limits | Labels are prompts and not proof of falsity. Certainty drops where a label depends on a subjective judgement of how the argument appears. |
Related techniques
- should follow TC1-19 Toulmin argument analysis: the argument must first be set out in explicit form
- should follow TC1-20 Argument mapping (argument diagramming): the map makes the links to be checked visible
Evidence and tag basis
Documented. The Stanford Encyclopedia records critiques: Hamblin on textbook treatment, Massey that no argument can be shown fallacious on purely logical grounds, and Johnson that the appearance condition is subjective. (Stanford Encyclopedia of Philosophy, Fallacies)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.