Skip to main content


Do problems in the conduct and reporting of Cochrane reviews limit the reliability of clinical conclusions?

Date and Location




Sunday 22 September 2013 - 15:30 - 17:00


Presenting author and contact person

Presenting author

Brian Alper

Contact person

Zbys Fedorowicz
Abstract text
Background: Cochrane reviews (CRs) are considered exemplars of quality. DynaMed is an evidence-based clinical reference which systematically summarizes CRs for point-of-care use. DynaMed summaries include a basis statement which reports the type of study supporting the conclusion and the major risk of bias (unless the conclusion is supported by level 1 evidence). This provides a unique opportunity to evaluate how often problems with the conduct and reporting of CRs hinder evidence assessment for clinical use. Objectives: To determine how often the level of evidence for conclusions derived from CRs is downgraded due to deficiencies in the CR conduct and reporting. Methods: We evaluated all the DynaMed basis statements referring to CRs. Each basis statement suggesting a downgrade due to deficiencies in the CR conduct or reporting (rather than deficiencies in the underlying evidence) was assessed against the CR by a senior editor from DynaMed and an experienced CR author to confirm such deficiencies. Results: On March 10, 2013 there were 5,142 basis statements in DynaMed referring to CRs. Thirty-nine (0.76%) were confirmed to have clinical conclusions graded level 2 or level 3 evidence because of deficiencies in the CR conduct or reporting. This analysis includes 4,743 unique CRs, of which 33 (0.7%) were found to have limitations in their conduct or reporting affecting reliability for clinical interpretation. In the corresponding DynaMed summaries we could neither clearly criticize the underlying evidence nor confirm high-quality evidence. Conclusions: CRs in general provide high quality assessment and synthesis of evidence which facilitates summarizing best current evidence for clinical decision-making. Implementation of the Methodological Expectations of Cochrane Intervention Reviews (MECIR) standards will improve compliance with conduct and reporting so that future reviews reliably match the expectations of both internal and external audiences to the Collaboration.