Skip to main content

Non-randomised studies in systematic reviews of intervention effectiveness: a content analysis of Cochrane Systematic Reviews

Date and Location




Sunday 22 September 2013 - 15:30 - 17:00


Presenting author and contact person

Presenting author

Jos Verbeek

Contact person

Jos Verbeek
Abstract text
Background: Non-randomized studies (NRS) are increasingly used as evidence for effectiveness of interventions in Cochrane reviews. How and why NRS are included has not been evaluated to date Objectives: to conduct an overview of practice of including NRS within the CC Methods: We included all Cochrane reviews that included one or more NRS. We conducted study selection and data collection in duplicate. We interpreted and categorized, via an iterative process, the reasons for including NRS, the way Risk of Bias was assessed and how this was influenced by other review characteristics Results: We included 202 reviews. All but one review included RCTs as well as NRS. NRS types that were specified were quasi-RCTs (45%), CCT (55%), controlled-before after studies (60%), interrupted time-series (52%), cohort-studies (37%), case-control studies (26%) and various other. Fifty percent of the reviews did not cite a reason for including NRS. For those that did, reasons could be divided into: RCTs are wanted but not available and RCTs are not needed because NRS provide good evidence. Unavailability was underpinned with: lack of RCTs based on a pilot search, randomization not feasible because of complex intervention, setting or aggregate level and randomization not ethical because of a vulnerable population. The claim that RCTs are not needed was supported by: restriction to RCTs is too dogmatic, NRS maximize the available data, NRS are needed to compare with RCTs and infrequent and long-term outcomes can only be studied in NRS. Risk of bias was based on EPOC-criteria in 38% the reviews. The rest used a variety of checklists and self-constructed tools. Conclusions: Most reviews don’t specify reasons for including NRS. Where they do, there is a wide variation in reasons that are not always supported by valid arguments. Risk of bias assessment of NRS also varies and needs urgent improvement.