Skip to main content

Assessing the quality of systematic reviews of non-randomized studies of intervention effects

Date and Location




Friday 20 September 2013 - 10:30 - 12:00
Contact persons and facilitators

Contact person

Beverley Shea


Beverley Shea
David Henry
Other contributors
First nameLast nameAffiliation and Country

First name


Last name


Affiliation and Country

University of Ottawa Heart Institute Canada

First name


Last name


Affiliation and Country

CIET Canada
Target audience

Target audience

Review authors, policy makers, researchers with an interest in bias and adverse effects

Is your workshop restricted to a specific audience or open to all Colloquium participants?


Level of knowledge required

Type of workshop

Type of workshop

Abstract text


Objectives: To illustrate limitations of applying quality assessment instruments to systematic reviews of non-randomized studies of treatment effects. Using AMSTAR, to discuss how risk of bias assessments with individual NRS can be used in evaluating systematic reviews that include them. Description: Over 60% of published systematic reviews now include non-randomized studies (NRS). Many are reviews of studies of intervention effects that are of interest to clinicians and policy makers. An increasing number of NRS are conducted in large databases or compilations of electronic health records. Such studies provide precise but potentially biased estimates of intervention effect sizes. Assessing the risk of bias of individual studies is a key step in conducting reviews. There are many quality assessment instruments available and Cochrane is adapting the Risk of Bias tool for use with NRS. How should such individual study quality inform our assessment of systematic reviews that include them? Do certain flaws invalidate the review? After a brief presentation group sessions will be held. Participants will use AMSTAR to assess systematic reviews that include NRS of varying quality; discuss how the quality of the component studies was assessed during conduct of the reviews and the impact this has on the review findings.