Skip to main content

Languages

Upgrading the quality of evidence from RCTs? Examples from published Summary of Findings tables

Date and Location

Session: 

O4.03.4

Date

Monday 23 September 2013 - 13:30 - 15:00
Presenting author and contact person

Presenting author

Miranda Langendam

Contact person

Miranda Langendam
Holger Schunemann
Abstract text
Background: Assessing the quality of the body of evidence (QoE) using GRADE criteria and justifying and documenting these assessments are highly desirable for new Cochrane Reviews as part of Summary of Findings (SoF) tables (MECIR standards). Although - in theory - possible, examples for upgrading a body of evidence from RCTs are elusive. Our experience in working with authors of systematic reviews and guideline developers is that interpretation of this guidance is often challenging. Objectives: To summarize if, and how Cochrane authors use upgrading the quality of RCT evidence in Cochrane reviews. Methods: We reviewed all SoF tables published in the Cochrane Library Issue 3, 2012 (n=502) and selected those tables in which the quality of RCT evidence was upgraded. These were studied in detail to see whether upgrading was appropriate (using the criteria of the GRADE JCE paper on upgrading). Results: In 3 SoF tables upgrading of downgraded RCTs was used explicitly, in 6 SoF tables implicitly. In 8 tables the QoE was upgraded because of a large magnitude of effect; in 1 because of a dose-response association. In all 9 tables risk of bias was the downgrading factor (unclear allocation concealment, incomplete outcome assessment, early stopping, no ITT analysis, outcome reporting bias), in 1 table in combination with imprecision and in 2 tables in combination with heterogeneity. Studying these SoF tables revealed that in all cases upgrading was not appropriate or at least questionable. In some SoF tables downgrading for imprecision was overlooked. Conclusions: Upgrading of RCTs affected by study limitations was considered appropriate by Cochrane authors in 2% of the reviewed SoF tables, mostly because of a large magnitude of effect. According to the current GRADE guidance on upgrading, the upgrading was at least questionable. More detailed guidance and worked out examples are needed. These will be presented at the Colloquium.