Investigating bias in network meta-analysis

Abstract

The advantages of network meta-analysis (NMA) have rendered it an increasingly popular statistical tool. The fact that methodology of NMA is relatively new implies the need for user-friendly software options to facilitate the appropriate conduct and reporting of the analysis. To aid the ease of presentation and interpretation of NMA, I developed new or modified existing graphical presentation tools, which I implemented in the STATA software. These graphs can be used to present the evidence base, assumptions and results from NMA making these outputs understandable to researchers that are less familiar with advanced statistical methods.An important threat for the validity of NMA findings is the presence of inconsistency and heterogeneity, which can be seen as differences in the potential effect modifiers within and across comparisons. I investigated the impact of five trial characteristics (four RoB components and study precision) on treatment effect estimates using the methodology of network meta-regression and meta-epidemiology on a collection of 32 star-shaped networks. In this analysis, inadequate random sequence generation, allocation concealment, blinding of participants and blinding of outcome assessors was not found to influence substantially the estimated relative effects in trials. I found a strong association between study precision and treatment effects for dichotomous outcomes using the same collection of star-shaped networks. I also exemplified several assumptions for the direction of bias using two full networks evaluating secondary outcomes. The direction of bias did not favor consistently the newer or older interventions in all studies. These findings imply that researchers should routinely consider the differences in precision across studies in the evaluation of transitivity and inconsistency. However, other trial characteristics associated with study size and relative effects may operate as well.It is clear so far that drawing reliable conclusions from NMA is often a challenging issue. Thus, accuracy in conduct of the analysis, careful evaluation of the assumptions and transparency in reporting are the fundamentals that should underlie any NMA publication.