Skip to main content

Languages

Risk of bias assessment of studies included in multiple Cochrane reviews: agreement between different reviews.

Date and Location

Session: 

O2.05.1

Date

Saturday 21 September 2013 - 13:30 - 15:00

Location

Presenting author and contact person

Presenting author

gabriel rada

Contact person

gabriel rada
Solange Rivera
Daniel Capurro
Abstract text
Background: Cochrane systematic reviews (SR) must include an assessment of the risk of bias (RoB) of included studies. Although there are numerous tools to assess the RoB of any given study, the Cochrane Collaboration endorses the RoB tool since 2008. Despite this standardization effort, the tool is based on subjective judgments and requires access to all relevant methodological information of each included study. Objectives: To assess whether different Cochrane reviews assessing the same included studies agree in their use of RoB criteria. Methods: We entered all Cochrane reviews and their corresponding included studies into the Epistemonikos database (www.epistemonikos.org). Epistemonikos links SRs to all their included studies. We selected a random sample of studies included in two or more Cochrane SR published since 2008. For each included study we extracted the RoB assessment provided in the corresponding section of the SR. This yielded at least two RoB assessments per study. Finally, we calculated agreements between the assessments made by different reviews for the same RoB item. Results: Our search was conducted in January 2013. We identified 5765 studies included in two or more Cochrane SRs from which we selected a random sample of 40 records. The RoB item most consistently assessed was ‘randomization’, assessed in all studies. For 33 studies (82.5%) its assessment was concordant. Concordance was observed in 72.5% of studies for ‘allocation concealment’, and 67.5% for ‘attrition bias’. On the other side of the spectrum, concordance was only 20% for performance bias and detection bias. Complete results, including a larger sample will be presented at the Colloquium Conclusions:The Cochrane Collaboration has made efforts to standardize risk of bias assessment of included studies across SRs. However, our analysis shows that RoB assessment is still inconsistent and might lead to equivocal appraisal of the available evidence.