Table 1.
Characteristics of 387 systematics reviews in the context of missing outcome data
| Characteristic | Levels | Total | 2009 (n = 21) |
2010 (n = 18) |
2011 (n = 48) |
2012 (n = 50) |
2013 (n = 65) |
2014 (n = 92) |
2015 (n = 68) |
2016 (n = 14) |
2017 (n = 11) |
|||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| n | % | |||||||||||||||||||||
| Addressing and handling missing outcome data | ||||||||||||||||||||||
| Study protocol | Registered | 24 | 6.2 | 0 | 0.0 | 0 | 0.0 | 1 | 4.2 | 1 | 4.2 | 6 | 25.0 | 8 | 33.3 | 8 | 33.3 | 0 | 0.0 | 0 | 0.0 | |
| Not registered but published | 14a | 3.6 | 1 | 7.1 | 0 | 0.0 | 2 | 14.3 | 2 | 14.3 | 6 | 42.9 | 1 | 7.1 | 2 | 14.3 | 0 | 0.0 | 0 | 0.0 | ||
| Mentioned but not available | 66 | 17.1 | 4 | 6.1 | 3 | 4.5 | 12 | 18.2 | 6 | 9.1 | 10 | 15.2 | 17 | 25.8 | 9 | 13.6 | 2 | 3.0 | 3 | 4.5 | ||
| Explicitly mentioned that there is no protocol | 4 | 1.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 25.0 | 0 | 0.0 | 3 | 75.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
| Protocol not mentioned | 279 | 72.1 | 16 | 5.7 | 15 | 5.4 | 33 | 11.8 | 40 | 14.3 | 43 | 15.4 | 63 | 22.6 | 49 | 17.6 | 12 | 4.3 | 8 | 2.9 | ||
| If protocol is available (37 SRs), the MOD strategy was determined already in the protocolb | Yes, using MOD as secondary outcome | 5 | 13.5 | 1 | 0.2 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 0.6 | 1 | 0.2 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | |
| Yes, in primary analysis using ITT with clarifications | 7 | 18.9 | 1 | 14.3 | 0 | 0.0 | 1 | 14.3 | 1 | 14.3 | 2 | 28.5 | 1 | 14.3 | 1 | 14.3 | 0 | 0.0 | 0 | 0.0 | ||
| Yes, in primary analysis using ITT without clarifications | 2 | 5.4 | 0 | 0.0 | 0 | 0.0 | 1 | 50.0 | 0 | 0.0 | 1 | 50.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
| Yes, in primary analysis by excluding trials with MOD | 1 | 2.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 100 | 0 | 0.0 | 0 | 0.0 | ||
| Yes, in primary analysis by excluding participants with MOD | 1 | 2.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 100 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
| No MOD strategy was determined | 23 | 62.2 | 0 | 0.0 | 0 | 0.0 | 1 | 4.3 | 1 | 4.3 | 7 | 30.4 | 7 | 30.4 | 7 | 30.4 | 0 | 0.0 | 0 | 0.0 | ||
| If protocol is available with a MOD strategy (14 SRs), the strategy defined in the protocol differed from that applied in the review | No, the authors adhered to the protocol | 12 | 85.7 | 0 | 0.0 | 0 | 0.0 | 2 | 16.7 | 1 | 8.3 | 1 | 8.3 | 4 | 33.3 | 4 | 33.3 | 0 | 0.0 | 0 | 0.0 | |
| MOD were not addressed eventually | 2 | 14.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 50.0 | 1 | 50.0 | 0 | 0.0 | 0 | 0.0 | ||
| The authors explained (in the protocol or review) what they considered as MOD | Yes, an explicit definition is provided | 63 | 16.3 | 4 | 6.3 | 7 | 11.1 | 13 | 20.6 | 4 | 6.3 | 12 | 19.0 | 11 | 17.5 | 8 | 12.7 | 3 | 4.8 | 1 | 1.6 | |
| No explanation is provided | 324 | 83.7 | 17 | 5.2 | 11 | 3.4 | 35 | 10.8 | 46 | 14.2 | 53 | 16.4 | 81 | 25.0 | 60 | 18.5 | 11 | 3.4 | 10 | 3.1 | ||
| The review explicitly reported whether LOCF was employed or not in the included trials | Yes, and they distinguished between LOCF and completely MOD | 5 | 1.3 | 0 | 0.0 | 1 | 20.0 | 1 | 20.0 | 0 | 0.0 | 2 | 40.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 20.0 | |
| Yes, but they didn’t distinguish between LOCF and completely MOD | 18 | 4.6 | 0 | 0.0 | 1 | 5.6 | 4 | 22.2 | 2 | 11.1 | 3 | 16.7 | 6 | 33.3 | 1 | 5.6 | 0 | 0.0 | 1 | 5.6 | ||
| No | 364 | 94.1 | 21 | 5.8 | 16 | 4.4 | 43 | 11.8 | 48 | 13.2 | 60 | 16.5 | 86 | 23.6 | 67 | 18.4 | 14 | 3.8 | 9 | 2.5 | ||
| There is no evidence that MOD exist in the included trials for the primary outcomes | MOD are present in the network | 273 | 70.5 | 10 | 3.7 | 14 | 5.1 | 35 | 12.8 | 29 | 10.6 | 44 | 16.1 | 76 | 27.8 | 44 | 16.1 | 13 | 4.8 | 8 | 2.9 | |
| Yes | 1 | 0.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 100 | 0 | 0.0 | 0 | 0.0 | ||
| Nothing mentioned | 113 | 29.2 | 11 | 9.7 | 4 | 3.5 | 13 | 11.5 | 21 | 18.6 | 21 | 18.6 | 16 | 14.2 | 23 | 20.4 | 1 | 0.9 | 3 | 2.7 | ||
| If the review explicitly reported the presence of MOD (273 SRs), the strategy described in the Methods section to address MOD isb | by excluding participants with MOD from the analyses | 4 | 1.5 | 0 | 0.0 | 0 | 0.0 | 1 | 25.0 | 1 | 25.0 | 0 | 0.0 | 1 | 25.0 | 0 | 0.0 | 0 | 0.0 | 1 | 25.0 | |
| using ITT in the primary analysis with clarifications | 25 | 9.2 | 2 | 8.0 | 1 | 4.0 | 3 | 12.0 | 3 | 12.0 | 4 | 16.0 | 8 | 32.0 | 3 | 12.0 | 0 | 0.0 | 1 | 4.0 | ||
| using ITT in the primary analysis without further clarifications | 84 | 30.8 | 1 | 1.2 | 1 | 1.2 | 10 | 11.9 | 10 | 11.9 | 20 | 23.8 | 23 | 27.4 | 14 | 16.7 | 3 | 3.6 | 2 | 2.3 | ||
| using dropout as a secondary outcome | 61 | 22.3 | 4 | 6.6 | 4 | 6.6 | 12 | 19.7 | 4 | 6.6 | 14 | 22.9 | 11 | 18.0 | 7 | 11.5 | 3 | 4.9 | 2 | 3.2 | ||
| using sensitivity analysis | by excluding studies based on a MOD rate threshold | 6 | 2.2 | 2 | 33.3 | 0 | 0.0 | 1 | 16.7 | 0 | 0.0 | 1 | 16.7 | 1 | 16.7 | 1 | 16.7 | 0 | 0.0 | 0 | 0.0 | |
| by excluding participants with MOD | 6 | 2.2 | 0 | 0.0 | 1 | 16.7 | 1 | 16.7 | 1 | 16.7 | 0 | 0.0 | 2 | 33.3 | 1 | 16.7 | 0 | 0.0 | 0 | 0.0 | ||
| other | 4 | 1.5 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 25.0 | 0 | 0.0 | 0 | 0.0 | 2 | 50.0 | 1 | 25.0 | 0 | 0.0 | ||
| using subgroup analysis | 6 | 2.2 | 0 | 0.0 | 1 | 16.7 | 0 | 0.0 | 0 | 0.0 | 3 | 50.0 | 2 | 33.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
| using meta-regression analysis | 3 | 1.1 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 33.3 | 2 | 66.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
| Not mentioned | 120 | 44.0 | 4 | 3.3 | 9 | 7.5 | 13 | 10.8 | 11 | 9.2 | 14 | 11.7 | 34 | 28.3 | 24 | 20.0 | 7 | 5.9 | 4 | 3.3 | ||
| In case of ‘intention-to-treat analysis’ (109 SRs), did the authors extract data as reported in the trials or they applied ITT de novo? | Data extracted as reported in the trials | 46 | 42.2 | 0 | 0.0 | 1 | 2.2 | 3 | 6.5 | 3 | 6.5 | 11 | 23.9 | 15 | 32.6 | 8 | 17.4 | 3 | 6.5 | 2 | 4.3 | |
| Intention-to-treat analysis de novo | 31 | 28.4 | 1 | 3.2 | 1 | 3.2 | 6 | 19.4 | 3 | 9.7 | 6 | 19.4 | 11 | 35.5 | 2 | 6.5 | 0 | 0.0 | 1 | 3.2 | ||
| Combination of the aforementioned | 13 | 12.0 | 1 | 7.7 | 0 | 0.0 | 3 | 23.1 | 4 | 30.8 | 1 | 7.7 | 1 | 7.7 | 3 | 23.1 | 0 | 0.0 | 0 | 0.0 | ||
| Unclear | 19 | 17.4 | 1 | 5.3 | 0 | 0.0 | 1 | 5.3 | 3 | 15.8 | 6 | 31.6 | 4 | 21.1 | 4 | 21.1 | 0 | 0.0 | 0 | 0.0 | ||
| Bias due to MOD was evaluated | Yes, using a specific tool | 198c | 51.2 | 6 | 3.0 | 8 | 4.0 | 21 | 10.6 | 15 | 7.6 | 35 | 17.7 | 52 | 26.3 | 43 | 21.7 | 10 | 5.1 | 8 | 4.0 | |
| Yes, probably but results are not displayed | 72 | 18.6 | 5 | 6.9 | 3 | 4.2 | 9 | 12.5 | 11 | 15.3 | 14 | 19.4 | 20 | 27.8 | 9 | 12.5 | 0 | 0.0 | 1 | 1.4 | ||
| No, but other domains were evaluated | 25 | 6.5 | 2 | 8.0 | 1 | 4.0 | 4 | 16.0 | 4 | 16.0 | 4 | 16.0 | 6 | 24.0 | 3 | 12.0 | 0 | 0.0 | 1 | 4.0 | ||
| No evaluation of risk of bias | 92 | 23.8 | 8 | 8.7 | 6 | 6.5 | 14 | 15.2 | 20 | 21.7 | 12 | 13.0 | 14 | 15.2 | 13 | 14.1 | 4 | 4.3 | 1 | 1.1 | ||
| Bias due to MOD was evaluated explicitly | No explicit evaluation | 181 | 46.8 | 6 | 3.3 | 8 | 4.4 | 21 | 11.6 | 15 | 8.3 | 33 | 18.2 | 46 | 25.4 | 38 | 21.0 | 8 | 4.4 | 6 | 3.3 | |
| With justification of judgments | 18c | 4.6 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 17.6 | 6 | 35.3 | 4 | 23.5 | 2 | 11.8 | 2 | 11.8 | ||
| Results on bias due to MOD are not displayed | 15 | 3.9 | 3 | 20.0 | 2 | 13.3 | 2 | 13.3 | 5 | 33.3 | 1 | 6.7 | 0 | 0.0 | 2 | 13.3 | 0 | 0.0 | 0 | 0.0 | ||
| No evaluation of bias due to MOD | 25 | 6.5 | 2 | 8.0 | 1 | 4.0 | 4 | 16.0 | 4 | 16.0 | 4 | 16.0 | 6 | 24.0 | 3 | 12.0 | 0 | 0.0 | 1 | 4.0 | ||
| Only an overall score is provided for each trial | 37 | 9.6 | 1 | 2.7 | 1 | 2.7 | 6 | 16.2 | 6 | 16.2 | 7 | 18.9 | 12 | 32.4 | 4 | 10.8 | 0 | 0.0 | 0 | 0.0 | ||
| Results on the risk of bias evaluation are not presented | 19 | 4.9 | 1 | 5.3 | 0 | 0.0 | 1 | 5.3 | 0 | 0.0 | 5 | 26.3 | 8 | 42.1 | 3 | 15.8 | 0 | 0.0 | 1 | 5.3 | ||
| No evaluation of risk of bias | 92 | 23.8 | 8 | 8.7 | 6 | 6.5 | 14 | 15.2 | 20 | 21.7 | 12 | 13.0 | 14 | 15.2 | 13 | 14.1 | 4 | 4.3 | 1 | 1.1 | ||
| Acknowledging implications of missing outcome data | ||||||||||||||||||||||
| Among the reviews with MOD (273 SRs), the interpreted results accounted for MOD | Yes | 88 | 32.2 | 4 | 4.5 | 7 | 8.0 | 16 | 18.2 | 8 | 9.1 | 17 | 19.3 | 20 | 22.7 | 10 | 11.4 | 4 | 4.5 | 2 | 2.3 | |
| No | 185 | 67.8 | 6 | 3.2 | 7 | 3.8 | 19 | 10.3 | 21 | 11.4 | 27 | 14.6 | 56 | 30.3 | 34 | 18.4 | 9 | 4.9 | 6 | 3.2 | ||
| If the interpreted results accounted for MOD (88 SRs), MOD implications were reported inb | Abstract | 46 | 52.3 | 3 | 6.5 | 2 | 4.3 | 8 | 17.4 | 3 | 6.5 | 9 | 19.6 | 11 | 23.9 | 5 | 11.0 | 3 | 6.5 | 2 | 4.3 | |
| Results | 26 | 29.5 | 1 | 3.8 | 2 | 7.7 | 4 | 15.4 | 2 | 7.7 | 4 | 15.4 | 8 | 30.8 | 4 | 15.4 | 1 | 3.8 | 0 | 0.0 | ||
| Discussion | 74 | 84.1 | 4 | 5.4 | 5 | 6.8 | 13 | 17.6 | 6 | 8.1 | 16 | 21.6 | 18 | 24.3 | 7 | 9.5 | 3 | 4.0 | 2 | 2.7 | ||
| Conclusions | 11 | 12.5 | 0 | 0.0 | 1 | 9.1 | 2 | 18.2 | 1 | 9.1 | 2 | 18.2 | 2 | 18.2 | 1 | 9.1 | 0 | 0.0 | 2 | 18.2 | ||
| If the interpreted results accounted for MOD (88 SRs), they were discussed in the context of which NMA componentsb | NMA treatment effects | 84 | 95.5 | 4 | 4.8 | 7 | 8.3 | 15 | 17.8 | 7 | 8.3 | 16 | 19.0 | 20 | 23.8 | 10 | 11.9 | 4 | 4.8 | 1 | 1.2 | |
| Intervention ranking | 13 | 14.8 | 1 | 7.7 | 1 | 7.7 | 2 | 15.4 | 1 | 7.7 | 3 | 23.0 | 1 | 7.7 | 1 | 7.7 | 2 | 15.4 | 1 | 7.7 | ||
| Heterogeneity | 7 | 8.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 2 | 28.6 | 2 | 28.6 | 2 | 28.6 | 1 | 14.2 | 0 | 0.0 | 0 | 0.0 | ||
| Evidence consistency | 3 | 3.4 | 0 | 0.0 | 1 | 33.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 2 | 66.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
| What information the authors used to discuss the implications (88 SRs)b | Judgments from the risk of bias assessment | 10 | 11.4 | 1 | 10.0 | 1 | 10.0 | 1 | 10.0 | 1 | 10.0 | 1 | 10.0 | 5 | 50.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | |
| The comments on the quality of evidence in SoF table | 1 | 1.1 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 100 | ||
| Sensitivity analysis results | 16 | 18.2 | 1 | 6.2 | 1 | 6.2 | 2 | 12.5 | 2 | 12.5 | 1 | 6.2 | 4 | 25.0 | 4 | 25.0 | 1 | 6.2 | 0 | 0.0 | ||
| Subgroup analysis on a dropout-relevant characteristic | 4 | 4.5 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 3 | 75.0 | 1 | 25.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
| Meta-regression analysis using dropout as covariate | 3 | 3.4 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 33.3 | 2 | 66.7 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
| NMA results on dropout (as a secondary outcome) | 58 | 65.9 | 4 | 6.9 | 3 | 5.2 | 11 | 19.0 | 4 | 6.9 | 14 | 24.1 | 10 | 17.2 | 7 | 12.1 | 3 | 5.2 | 2 | 3.4 | ||
| Dropout prevalence | 11 | 12.5 | 0 | 0.0 | 2 | 18.2 | 4 | 36.4 | 1 | 9.1 | 1 | 9.1 | 2 | 18.2 | 1 | 9.1 | 0 | 0.0 | 0 | 0.0 | ||
| Reasons for dropout | 1 | 1.1 | 0 | 0.0 | 1 | 100 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | ||
| The strategy used to handle MOD in primary analysis | 2 | 2.3 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 | 1 | 50.0 | 1 | 50.0 | 0 | 0.0 | 0 | 0.0 | ||
Abbreviations: ITT, intention-to-treat analysis, LOCF last observation carried forward, MOD missing outcome data, NMA network meta-analysis, SoF summary of finding, SRs systematic reviews
aOne review explicitly mentioned that the protocol is available, but the provided link page could not be found
bMultiple selections have been applied
cOne systematic review clearly indicated that no incomplete outcome data exist in any of the included trials