Handle evidence
These techniques grade the reliability of a source and the credibility of its information, check a source against criteria such as currency, authority and purpose, read what other sources say about a publisher, and compare independent lines of evidence. Use them when sources vary in quality and the answer has to say how far each can be trusted.
29 techniques. Open one to see what it does, when it fits and what it needs from your material.
| ID | Technique | Purpose | Evidence status |
|---|---|---|---|
| HE1-01 | Admiralty system (source reliability and information credibility grading) | Assigns each report a two-character grade for source reliability (A to F) and information credibility (1 to 6) and produces an appraised evidence base. | Documented |
| HE1-02 | Quality of information check | Rates the completeness, accuracy and corroboration of each source and produces a source database with a statement of what is known and unknown. | Documented |
| HE1-03 | CRAAP test (currency, relevance, authority, accuracy, purpose) | Screens one source against five criteria (currency, relevance, authority, accuracy, purpose) and produces a checklist judgement for the evidence base. | Documented |
| HE1-04 | SIFT (stop, investigate the source, find better coverage, trace claims to the original; claim verification and fact-checking) | Moves from a claim to its publisher, to other coverage and to the original source, and produces a short credibility judgement with the original context located. | Practice |
| HE1-05 | Lateral reading | Leaves the page to read about the publisher in other sources and produces a judgement of publisher credibility. | Unconfirmed |
| HE1-06 | Triangulation (source, method, investigator and theory triangulation; corroboration) | Compares independent lines of evidence on one question and produces a convergence table of agreement, disagreement and reasons. | Unconfirmed |
| HE1-07 | Evidence hierarchies and certainty grading (study design levels, risk-of-bias assessment, Grading of Recommendations Assessment, Development and Evaluation or GRADE) | Ranks study designs and rates confidence in a body of evidence from high to very low, and produces a certainty rating with reasons for downgrading. | Documented |
| HE1-08 | Systematic review | Collects all eligible studies by a pre-specified, reproducible search, appraises and synthesises them, and produces a screening log, an evidence table and a synthesis. | Documented |
| HE1-09 | Rapid review | Runs a systematic review with streamlined steps to meet a deadline and produces a narrative synthesis that states the shortcuts taken. | Documented |
| HE1-10 | Scoping review (evidence mapping) | Maps the extent, types and gaps of evidence on a broad topic without judging the strength of findings, and produces a charted table. | Unconfirmed |
| HE1-11 | Meta-analysis | Combines effect estimates from separate studies into a weighted average with a confidence interval and produces a forest plot, a pooled effect and heterogeneity statistics. | Documented |
| HE1-12 | Narrative, thematic and qualitative evidence synthesis (including meta-ethnography) | Combines findings across studies in words or themes when pooling numbers is not appropriate, and produces a structured narrative or theme set linked back to studies. | Documented |
| HE1-13 | Reflexive thematic analysis (Braun and Clarke) | Identifies and interprets patterns of meaning across qualitative data in six phases and produces themes with supporting extracts. | Documented |
| HE1-14 | Framework analysis (Framework Method) | Sorts coded qualitative data into a case by theme matrix and produces the matrix for reading across cases and themes. | Documented |
| HE1-15 | Content analysis (conventional, directed and summative) | Codes text into categories, then counts or interprets category use, and produces category frequencies or a category structure. | Unconfirmed |
| HE1-16 | Discourse analysis (critical discourse analysis) | Examines how language constructs topics and positions actors and produces an interpretive account of framing checked against quoted passages. | Unconfirmed |
| HE1-17 | Grounded theory coding (open, axial and selective coding; constant comparison) | Builds categories and a theory from data through open, axial and selective coding with constant comparison and produces codes, memos and a theory. | Unconfirmed |
| HE1-18 | Document and policy analysis | Reads documents systematically to extract facts, positions, changes or stated reasons using their provenance and purpose, and produces an extraction table or chronology. | Unconfirmed |
| HE1-19 | Bibliometric analysis (citation analysis, science mapping, co-citation, co-word analysis) | Counts and maps publications, authors and citations to show the structure and growth of a field and produces performance tables and network maps. | Unconfirmed |
| HE1-20 | Trend and time-series decomposition Also serves: Project forward. | Separates a series into trend, seasonal and irregular parts and produces plots and a fitted trend. | Unconfirmed |
| HE1-21 | Distribution, outlier and base-rate checks | Examines the shape, extremes and baseline frequency of data before a figure is interpreted and produces histograms, summary statistics, flagged outliers and a base rate. | Unconfirmed |
| HE-01 | Grey literature appraisal checklist (AACODS (Authority, Accuracy, Coverage, Objectivity, Date, Significance)) | Scores a non-peer-reviewed item on six criteria (authority, accuracy, coverage, objectivity, date, significance) and produces a completed checklist. | Documented |
| HE-02 | Critical appraisal checklist by study design (CASP (Critical Appraisal Skills Programme) checklists; JBI (Joanna Briggs Institute) checklists) | Answers design-specific questions on validity, results and relevance for one study and produces a completed checklist for the evidence base. | Documented |
| HE-03 | Appraisal of a systematic review (AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews 2); ROBIS (Risk Of Bias In Systematic reviews)) | Rates confidence in a systematic review with AMSTAR 2 (high to critically low) or concern in four domains with ROBIS, and produces a rating or domain judgements. | Documented |
| HE-04 | Survey disclosure completeness check (AAPOR (American Association for Public Opinion Research) disclosure elements) | Tests whether a survey report states the sponsor, question wording, population, sampling frame, mode, dates, sample size, margin of error and weighting, and produces a list of disclosed and missing items. | Documented |
| HE-05 | Margin of error and credibility interval check | Checks that a reported margin of error matches the sampling method and produces a judgement on whether the figure is valid. | Documented |
| HE-06 | Opt-in panel respondent quality check (bogus respondent screen) | Looks for implausible answers that show careless or fraudulent respondents inflating rare responses in an opt-in survey and produces a judgement of inflation risk. | Documented |
| HE-07 | Overview of reviews (umbrella review; tertiary review) | Uses systematic reviews as the unit of search, inclusion and analysis and produces a summary of reviews with a quality rating per review and an overlap map. | Documented |
| HE-08 | Living systematic review | Keeps a systematic review continuously updated as new studies appear and produces a versioned evidence summary and an update log. | Documented |