startthat.work home

CM-01 Sponsor versus non-sponsor comparison (meta-epidemiological sponsorship analysis)

Compares the share of favourable results and conclusions between sponsored and independently funded reports on one topic, producing a risk ratio or odds ratio.

CompareEvidence: DocumentedPrimary technique, runs in step 5.3Strongest statement: show a link between things

What it does

Compares the share of favourable results and conclusions between sponsored and independently funded reports on one topic. Artefact: a risk ratio or odds ratio. In health research the risk ratios were 1.27 for favourable efficacy results and 1.34 for favourable conclusions.

When to use it

Use when a user asks whether industry or sponsor funding is linked to more favourable findings in the reports on a topic.

When it does not fit

Needs enough independent reports on one question. The health studies are observational, so the result is an association. The harms result was not significant. No study of surveys or consultancy reports was found, so applying the result to them is an inference.

Result format

The result gives the number of sponsored and independently funded reports on the topic and the share of favourable results and favourable conclusions in each group. It reports a risk ratio or odds ratio for each of results and conclusions, and ends with a statement of what the ratio shows about sponsorship.

Requirements on the material

PropertyRequirement typeRequirement
Content typeAlways appliesrequires=text; requires=tabular
Unit of analysisAlways appliesrequires=source; requires=document
Number of sourcesAlways appliesrequires=two or more
Producer interestChanges the orderrequires=independent; requires=sponsored; requires=self-interested seller
Source dependenceChanges the orderprefers=independent
Appraises source credibilityApplies in some casesyes

Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Whether sources can be joined, Amount of material, Known data problems, Origin, Document genre, Language, Personal information, Security classification, Licence.

Minimum materialEnough independent and sponsored reports on one question to form a ratio, with a funder recorded for each report.
Additional sources needednone
Not for (rules)none
Not for (text)Needs enough independent reports on one question. Health studies are observational, so the result is an association. No study of surveys or consultancy reports was found, so applying the health result to them is an inference.

Choosing and ordering

Kind of techniquePrimary technique, runs in step 5.3
Phase taganalyse
Other phase tagsvalidate
Kind of findingdifference; risk
Question answeredhow much; what
Strongest statement it supportsshow a link between things
Needs as input
Makes as output
Must exist firstNone
Loop targetreturn to gather

Library checks by stakes

Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.

StakesCheckPerson involvedCounts as a different method
exploratoryspot checkoptionalno
decision-informingspot checkoptionalno
decision-informinghuman review sampleneeded; left out below high stakesno
decision-informingcode execution for arithmeticnoneno
high-stakesspot checkoptionalno
high-stakeshuman review sampleneededno
high-stakescode execution for arithmeticnoneno
high-stakessensitivity analysisnoneno
high-stakesindependent reviewneededno

Failure modes and limits

Model suitabilityhuman judgement required
Typical model failureAssigns a sponsor from the logo alone and treats independence as proof of quality.
Error sourcesinput data; source selection; coder or rater judgement
Certainty limitsCertainty drops because the evidence is observational (risk of bias), when few reports fall in one group (imprecision), and when the result is carried from health studies to other fields (indirectness).

Evidence and tag basis

Documented. A Cochrane review of 75 papers found the pattern was not explained by standard risk of bias items. An older review of 30 studies gave an odds ratio of 4.05. (Lundh et al. 2017, Cochrane abstract; Cochrane summary page, industry sponsorship and research outcome; Pharmaceutical industry sponsorship and research outcome and quality)

Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.