HE1-08 Systematic review
Collects all eligible studies by a pre-specified, reproducible search, appraises and synthesises them, and produces a screening log, an evidence table and a synthesis.
What it does
Collects all eligible studies by a pre-specified, reproducible search, appraises and synthesises them. Artefact: a flow diagram, evidence table and synthesis; read exclusion reasons and the risk-of-bias table.
When to use it
Use when you need the most complete and least biased answer the published evidence can give to a narrow question.
When it does not fit
The question is broad (mapping) or too few comparable studies exist.
Result format
The result has a flow diagram of studies found, screened, excluded and included, an evidence table with one row per included study, and a synthesis. It includes the exclusion reasons and a risk-of-bias table with one row per study.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Content type | Always applies | requires=text |
| Unit of analysis | Always applies | requires=document |
| Number of sources | Always applies | requires=two or more |
| Appraises source credibility | Applies in some cases | yes |
Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Whether sources can be joined, Amount of material, Known data problems, Producer interest, Source dependence, Origin, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | at least two eligible studies; a written protocol with eligibility criteria; two independent reviewers |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | The question is broad, so a map would serve better, or too few comparable studies exist to synthesise. |
Choosing and ordering
| Kind of technique | Preparation technique, runs in step 5.2 |
|---|---|
| Phase tag | gather |
| Other phase tags | verify and prepare; analyse; conclude |
| Kind of finding | pattern; gap |
| Question answered | what; how much |
| Strongest statement it supports | estimate the effect of an action |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to frame |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | spot check | optional | no |
| decision-informing | spot check | optional | no |
| decision-informing | human review sample | needed; left out below high stakes | no |
| high-stakes | spot check | optional | no |
| high-stakes | human review sample | needed | no |
| high-stakes | independent review | needed | no |
| high-stakes | second technique with different assumptions | none | conditional |
Failure modes and limits
| Model suitability | human judgement required |
|---|---|
| Typical model failure | Reports a search as exhaustive without a protocol, invents studies, or leaves out exclusion reasons. |
| Error sources | input data; source selection; coder or rater judgement |
| Certainty limits | Certainty drops with an incomplete search, a single screener, unpublished studies missed (reporting bias) and heterogeneous studies (inconsistency). |
Related techniques
- should follow FQ-01 PICO family (population, intervention, comparison, outcome; variants PICOT, PEO, PICo, SPIDER): a structured question fixes the eligibility criteria
- pairs with HE-02 Critical appraisal checklist by study design (CASP (Critical Appraisal Skills Programme) checklists; JBI (Joanna Briggs Institute) checklists): each included study needs design-specific appraisal
- pairs with HE1-10 Scoping review (evidence mapping): a scoping review can narrow a broad question first
Evidence and tag basis
Documented. The Cochrane Handbook treats finding unpublished studies as important for minimising bias and rates grey literature searching as highly desirable. (Cochrane Handbook, chapter 4)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.