PJ-23 Horizon scanning (environmental scanning)
Search systematically for early signs of change in sources outside the topic usual field, giving a signal database with source, date and rating as a collected evidence base.
What it does
Searches systematically for early signs of change in sources outside the topic's usual field. Artefact: a signal database with source, date and rating; read clusters of signals that recur across independent sources.
When to use it
Use when you need early warning of change from outside the usual sources and a recorded list of signals to review later.
When it does not fit
The sources are those already used for the existing view, or signals are collected without a later review of which came true.
Result format
The result is a signal database with one row per signal, giving the signal, its source, its date and its rating. It names the clusters of signals that recur across independent sources.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Structure | Applies in some cases | requires=unstructured; requires=semi-structured |
| Content type | Always applies | requires=text |
| Unit of analysis | Always applies | requires=document; requires=source |
| Number of sources | Always applies | requires=two or more |
| Source dependence | Changes the order | prefers=independent |
| Appraises source credibility | Applies in some cases | no |
Not restricted on: Variable types, Time structure, How the sample was chosen, Level of detail available, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | A scanning scope, a source list outside the usual field and a recording template. |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | The sources are those already used for the existing view, or signals are collected without a later review of which came true. |
Choosing and ordering
| Kind of technique | Preparation technique, runs in step 5.2 |
|---|---|
| Phase tag | gather |
| Other phase tags | analyse |
| Kind of finding | trend; gap; pattern |
| Question answered | what; what next |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to gather |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | spot check | optional | no |
| decision-informing | spot check | optional | no |
| decision-informing | provenance log | none | no |
| decision-informing | human review sample | needed; left out below high stakes | no |
| high-stakes | spot check | optional | no |
| high-stakes | provenance log | none | no |
| high-stakes | human review sample | needed | no |
| high-stakes | independent review | needed | no |
| high-stakes | second technique with different assumptions | none | conditional |
Failure modes and limits
| Model suitability | model plus code |
|---|---|
| Typical model failure | Records signals from the same sources as the existing view and counts repeated reports as independent signals. |
| Error sources | source selection; input data; language model |
| Certainty limits | Confidence drops when sources overlap with the existing view (source selection bias), when repeated reports are counted as independent (risk of bias), or when no later review of outcomes exists (indirectness). |
Related techniques
- pairs with PE-03 Near-duplicate detection (shingling; MinHash; locality-sensitive hashing; text reuse detection): near-duplicate detection stops one repeated report counting as several signals
- pairs with IS-06 Source dependence and copy detection (source independence analysis): copy detection tests whether recurring signals come from independent sources
- should follow PE-01 Corpus construction and sampling (document selection; search string design; sampling frame design): a defined corpus and search string set the scope of the scan
Evidence and tag basis
Documented. Listed in the UK Futures Toolkit as spotting early signs of change; no accuracy evidence found. (GOV.UK Futures Toolkit case study)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.