Skip to main content
. 2015 Jul 23;24(4):266–274. doi: 10.1002/mpr.1471

Table 2.

Receiver operating characteristic (ROC) model statistics

Age Group Variable Cut pointa CAPA/PAPA estimateb Modeled estimateb Biasb FPc (%) FNc (%) Sensd Specd PPVe NPVe AUCdf AUCcf
Model one = four‐level impact item (0–3) predictor
4–11 MDI 1 21.55 36.88 15.33 25.09 9.76 0.547 0.680 0.320 0.845 0.614 0.714
4–11 MDI 2 21.55 5.15 −16.40 1.02 17.43 0.191 0.987 0.801 0.816 0.589
4–11 MDSI 2 8.81 5.15 −3.67 1.35 5.02 0.430 0.985 0.737 0.947 0.708 0.806
12–17 MDI 1 21.00 30.20 9.20 11.32 2.12 0.899 0.857 0.625 0.970 0.878 0.803
12–17 MDI 2 21.00 7.32 −13.68 1.78 15.46 0.264 0.977 0.756 0.833 0.621
12–17 MDSI 2 9.99 7.32 −2.67 1.93 4.60 0.539 0.979 0.736 0.950 0.759 0.866
Model two = SDQ score (0–10) predictor
4–11 MDI 3 21.55 25.07 3.52 12.37 8.85 0.589 0.842 0.507 0.882 0.716 0.764
4–11 MDSI 4 8.81 9.32 0.50 7.23 6.73 0.237 0.921 0.224 0.926 0.579 0.797
12–17 MDI 4 21.00 26.13 5.13 13.54 8.41 0.599 0.829 0.482 0.886 0.714 0.813
12–17 MDSI 5 9.99 11.04 1.05 5.41 4.35 0.564 0.940 0.510 0.951 0.752 0.862
Model three=SDQ score (0–10) and impact item (0 –3) predictors
4–11 MDI 21.55 25.18 3.63 12.47 8.85 0.589 0.841 0.505 0.882 0.715 0.785
4–11 MDSI 8.81 7.97 −0.85 3.77 4.62 0.476 0.959 0.526 0.950 0.717 0.834
12–17 MDI 21.00 21.14 0.14 7.11 6.97 0.668 0.910 0.664 0.912 0.789 0.847
12–17 MDSI 9.99 7.93 −2.06 2.15 4.21 0.579 0.976 0.729 0.954 0.777 0.913
Model four=subset of up to five individuaL SDQ item scores (0–2) and impact item (0–3) predictors
4–11 MDI9 21.55 21.12 −0.43 7.59 8.01 0.628 0.903 0.641 0.898 0.766 0.800
4–11 MDSI 8.81 8.92 0.11 4.77 4.66 0.471 0.948 0.466 0.949 0.710 0.873
12–17 MDI 21.00 21.24 0.24 7.84 7.60 0.638 0.901 0.631 0.904 0.770 0.851
12–17 MDSI 9.99 10.02 0.03 3.34 3.31 0.669 0.963 0.667 0.963 0.816 0.910

AUCc=area under ROC curve (continuous predictor); AUCd=area under ROC curve (dichotomous predictor); FN=false negative; FP=false positive; MDI=mental disorder with impairment; MDSI=mental disorder with severe impairment; NPV=negative predictive value; CAPA/PAPA= child and adolescent psychiatric assessment (CAPA) or preschool age psychiatric assessment (PAPA); PPV=positive predictive value; ROC=receiver operating characteristic, Sens=sensitivity, Spec=specificity.

a

For models with a single predictor, a cut point of x indicates that a score ≥ x will result in a positive MDI or MDSI prediction; for models with multiple predictors, the cut point is not given (see Table 2, footnotes 7 and 8).

b

CAPA/PAPA Estimate and Modeled Estimate refer to prevalence estimates of MDI or MDSI based on CAPA/PAPA determinations and model predictions, respectively. Bias is the difference between modeled and CAPA/PAPA estimates.

c

FP and FN are percentages of weighted false positive and false negative counts, respectively.

d

Sens (i.e. sensitivity) is the weighted proportion of CAPA/PAPA MDI or MDSI positive cases also predicted MDI or MDSI positive (i.e. positive “hit” rate). Spec (i.e. specificity) is the weighted proportion of CAPA/PAPA MDI or MDSI negative cases also predicted MDI or MDSI negative (i.e. 1 – “false alarm” rate).

e

PPV is the weighted proportion of predicted MDI or MDSI positive cases that are also CAPA/PAPA MDI or MDSI positive. NPV is the weighted proportion of predicted MDI or MDSI negative cases that are also CAPA/PAPA MDI or MDSI negative.

f

AUCc is the concordance between the model (with continuous predictor) and CAPA/PAPA MDI or MDSI, and AUCd is the concordance between the model (with predictor dichotomized by the cut point) and CAPA/PAPA MDI or MDSI; AUCd = the average of sensitivity and specificity.

See Table 3 for cut points determined for combination of impact item score and SDQ score.

Cut point not given, because it would involve multiway tables or a detailed model description and cut point in terms of predicted probabilities.

Example formula: MDI (4–11 years) = if 1/(1 + exp[–(–3.7309 + 2.8261*Obed + 1.1503*Worry + 2.0109*Unhappy + 0.2049*Atten + 0.6565*Impact]) ≥ 0.46940 where Obed(iance), Worry, Unhappy, Adults, and Atten(tion) refer to the five items used in the abbreviated SDQ (this may require recoding of questionnaire raw scores) and impact refers to a sixth SDQ item. Other formulas available upon request.