Skip to main content


Empirical evidence about inconsistency in complex networks of interventions

Date and Location




Monday 23 September 2013 - 13:30 - 15:00


Presenting author and contact person

Presenting author

Areti Angeliki Ve...

Contact person

Areti Angeliki Ve...
Abstract text
Background: Network meta-analysis relies on the agreement between direct and indirect evidence defined as consistency. Empirical evidence about the prevalence of inconsistency is limited to simple loops of evidence about three interventions. Objectives: To evaluate empirically the prevalence of inconsistency in full complex networks and explore factors that might control its statistical detection. Methods: We evaluated inconsistency in 40 published networks with dichotomous outcomes (303 loops of evidence). We employed four different approaches: 1) loop-specific: we evaluated each loop in the network separately by contrasting direct and indirect estimates 2) separating direct and indirect evidence (SIDE): we evaluated the agreement between a particular comparison and the rest of the network 3) Lu and Ades model: we jointly assessed all possible inconsistencies in the network to obtain an omnibus test 4) Design-by-Treatment interaction model (DbT): we evaluated the agreement between estimates from different study designs in the network. In each approach we assessed the assumption of consistency using odds ratio, risk ratio, and risk difference, and we considered different estimators for the heterogeneity. Results: The loop-specific method showed that inconsistency was prevalent in up to 10% of the tested loops. Loops including comparisons informed by a single study were more likely to show inconsistency. The SIDE method showed that about 10% of the tested comparisons were inconsistent. The Lu and Ades model yielded 2 (5%) inconsistent networks in total. The DbT model suggested inconsistency in 13% of the networks. Important heterogeneity was associated with a small decrease in statistical inconsistency. Different effect measures had no important impact on the detection of inconsistency. Conclusions: Inconsistency can occur in one in ten of the loops and in one in eight networks. A sensitivity analysis for the estimation of heterogeneity might be needed before reaching a conclusion about consistency.