Skip to main content

Languages

Are claims of interventional benefit in clinical trial abstract conclusions justifiable? A case report of pharmacological and complementary therapies for chronic asthma

Date and Location

Session: 

P3.036

Date

Sunday 22 September 2013 - 10:30 - 12:00

Location

Presenting author and contact person

Presenting author

Rachel Toone

Contact person

Rachel Toone
Abstract text
Background Conclusions in abstracts of randomised controlled trials (RCTs) should accurately reflect study results, as people may only read this section of the report. Authors sometimes use strategies (“spin”) to suggest therapies are beneficial, even if results suggest otherwise. We examined how often, when abstracts of RCTs of interventions for asthma suggest an intervention has benefit, this claim is justifiable. To suggest interventional benefit, we felt studies must recruit the pre-specified sample size, and demonstrate statistically significant between-group differences for the primary outcome (p<0.05). Methods Studies were identified from Cochrane reviews of chronic asthma therapies. We included RCTs, of pharmacological and non-pharmacological/complementary interventions, in which the abstract conclusion suggested interventional benefit. The abstract conclusion of each RCT was categorised:(1)suggesting benefit without uncertainty; (2)suggesting benefit with uncertainty (eg more studies required); (3)not suggesting benefit. We identified whether the trial report presented a required sample size, primary outcome, and between-group comparison for this. For abstracts suggesting benefit without uncertainty, we evaluated whether the claim was (1) justified (sufficient participants, and primary outcome between-group comparison p<0.05); (2)unjustified (not all criteria met); (3)non-evaluable (insufficient information). As an exploratory evaluation of “spin”, we evaluated how often, when abstract conclusions suggest benefit with uncertainty, this claim appeared based on secondary outcome/within-group comparisons. Results Of 153 abstracts screened, 42 full RCT reports were evaluated. 26/42 abstracts suggested benefit with no uncertainty. This was justified in 11/26, unjustified in 3/26, and non-evaluable in 12/26. 16/42 abstracts suggested benefit with uncertainty. In 11/16, the suggested benefit was felt to be based on secondary outcome/within-group comparisons (ie “spin”). Non-evaluable claims of benefit, and “spin”, were particularly common in RCTs of non-pharmacological/complementary therapies. Conclusion When abstracts of RCT reports suggest benefit of experimental treatment, the full report frequently includes insufficient information to justify this claim. Spin is commonplace in RCT abstracts.