Skip to main content


Use of outcome data hierarchies to deal with multiplicity in trials: survey of systematic reviews

Date and Location




Sunday 22 September 2013 - 15:30 - 17:00


Presenting author and contact person

Presenting author

Matthew Page

Contact person

Matthew Page
Abstract text
Background: For a particular outcome in a systematic review, there may be a multiplicity of data available in the trial reports (e.g. multiple scales, time points, and analyses). Multiplicity can potentially lead to data driven decisions regarding which data to include. Pre-specified decision rules known as ‘outcome data hierarchies’ (e.g. listing eligible scales in order of preferred inclusion) are a suggested method to deal with multiplicity. Objective: To investigate how often systematic reviewers pre-specify outcome data hierarchies and the impact of such hierarchies on the extent of data available for inclusion in a meta-analysis. Methods: Cochrane and non-Cochrane systematic reviews published from January 2010 to January 2012 were randomly sampled. All trial data that were compatible with (i) the review definition of the first meta-analysed continuous outcome, and (ii) the hierarchies reported in the review protocol, where available, were extracted. The proportion of (i) trials with multiplicity, (ii) reviewers pre-specifying hierarchies, and (iii) reviews including at least one trial with multiplicity, were quantified. Results: Forty-four reviews including 276 trials were evaluated. Multiplicity was present in half the trials (95% confidence interval (CI) 44% to 57%); multiple scales and intervention groups were most common. In reviews with a published protocol (N=21), hierarchies were pre-specified in relation to the following: intervention groups (0%); scales (19%); time points (62%); intention-to-treat versus alternative analyses (71%); and final versus change from baseline values (29%). Reviews with at least one pre-specified hierarchy had a lower chance of including a trial with multiplicity compared to reviews with no pre-specified hierarchies (9/17 versus 23/27; risk difference -0.32, 95%CI -0.59 to -0.05). Conclusion: Use of outcome data hierarchies varies across systematic reviews. Pre-specifying hierarchies can reduce the number of trial effect estimates available for inclusion in a meta-analysis, and prevent selective inclusion based on the results.