Table 3. Examples of pre-specified acceptable validity levels.
| Source | Indicator types | Metric and Level |
|---|---|---|
| Ronsmans 1996 [54] | Maternal morbidity, obstetric complications | Fairly accurate if sensitivity and specificity >80%; high specificity is very important for rare outcomes to limit over-reporting of actual prevalence |
| Liu et al. 2013 [45] | Coverage and content of antenatal, delivery and postnatal care | Sensitivity/specificity: • Low <0.33 • Moderate 0.33–0.66 • High >0.66 AUC–overall validity “high” if AUC>0.67 (otherwise moderate/low) Population-level bias • Small if 0.8<TAP ratio<1.2 • Moderate 0.5<TAP ratio<1.5 • Large <0.5 TAP ratio >1.5 |
| Stanton et al. 2013 [44] | Maternal and newborn health interventions during peripartum period in health facilities | Acceptable if AUC>0.60 or IF 0.75–1.25 (Suggest indicators warranting incorporation into population-level surveys should meet both criteria- individual and population level validity) |
| Blanc et al. 2016 [40], McCarthy et al. 2016 [43] | Quality of maternal and newborn health care during childbirth | Individual-level accuracy measured by AUC • High: AUC>0.7 • Moderate: 0.6<AUC<0.7 • Low: AUC<0.6 Degree of bias measured by IF • Low 0.75<IF<1.25 • Moderate 0.5<IF<1.5 • Large IF<0.5 | IF>1.5 High overall performance: high AUC + low IF Reliability (decay in accuracy between baseline taken during exit interview and at 13–15 months post delivery)–Phi coefficient (rphi) • Poor if <0.4, Moderate between 0.4 and 0.6; High 0.6–0.8; > = 0.8 almost perfect agreement. |
| Blanc et al. 2016 [60] | Skilled birth attendant and key elements of maternal, intrapartum, newborn and immediate postnatal care among women with vaginal deliveries | Benchmarks of validity: • AUC> = 0.6 • 0.75<IF<1.25 Overall acceptable indicator performance: both AUC and IF benchmarks met |
| Munos et al. [42], Chang et al. 2018 [61] | Coverage indicators | AUC≥0.70 |
| IF 0.75–1.25 | ||
| (study authors acknowledge these are arbitrary) |
AUC: Area under the receiver operating characteristic curve; TAP: test to actual positive ratio; IF: inflation factor.