CH-14 Structured brainstorming
Generates and clusters many ideas about a focal question before any critique and produces clustered, prioritised ideas.
What it does
Generates and clusters many ideas about a focal question before any critique. Artefact: clustered, prioritised ideas; read the outlier ideas that were set aside.
When to use it
Use when you need a wide set of ideas, causes or options on a focal question before narrowing.
When it does not fit
Individuals cannot escape their own biases, and productivity falls after about 60 to 90 minutes.
Result format
The result is a set of ideas about the focal question, grouped into clusters with a name for each cluster, and a prioritised order of the ideas. It also lists the outlier ideas that were set aside, so they can be read.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Unit of analysis | Always applies | requires=topic |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Structure, Content type, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | A focal question and a group of four to twelve people, with 60 to 90 minutes. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | Individuals cannot escape their own biases, and productivity falls after about 60 to 90 minutes. |
Choosing and ordering
| Kind of technique | Question technique, runs in step 5.1 |
|---|---|
| Phase tag | frame |
| Other phase tags | analyse |
| Kind of finding | pattern; gap |
| Question answered | what; which option |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | proceed with limit stated |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | assumption and limitation statement | none | no |
| decision-informing | assumption and limitation statement | none | no |
| decision-informing | independent review | needed; left out below high stakes | no |
| high-stakes | assumption and limitation statement | none | no |
| high-stakes | independent review | needed | no |
| high-stakes | expert review | needed | no |
Failure modes and limits
| Model suitability | human judgement required |
|---|---|
| Typical model failure | Produces a long list of similar ideas from one view and clusters them by wording and not by meaning. |
| Error sources | input data; coder or rater judgement; language model |
| Certainty limits | No effectiveness evidence was read and the sources give different group sizes; idea sets reflect the participants and carry risk of bias. |
Related techniques
- pairs with CH-15 Group process safeguards (everyone speaks once before anyone speaks twice, mindguarding): Group safeguards keep individual views independent before ideas are pooled
- pairs with TC1-02 Fishbone diagram (Ishikawa diagram, cause and effect diagram): A fishbone diagram groups brainstormed causes of one problem
Evidence and tag basis
Documented. The two sources give different group sizes; no effectiveness evidence was read. (CIA Tradecraft Primer (2009); UK Red Teaming Handbook)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.