. Author manuscript; available in PMC: 2016 Dec 1.

Published in final edited form as: Int J Methods Psychiatr Res. 2015 Jul 23;24(4):266–274. doi: 10.1002/mpr.1471

Table 2. Receiver Operating Characteristic Model Statistics.

Model One = Four-Level Impact Item (0 to 3) Predictor
Age Group	Variable	Cut Point¹	P/CAPA Estimate²	Modeled Estimate²	Bias²	FP³ (Percent)	FN³ (Percent)	Sens⁴	Spec⁴	PPV⁵	NPV⁵	AUCd⁶	AUCc⁶
4–11	MDI	1	21.55	36.88	15.33	25.09	9.76	0.547	0.680	0.320	0.845	0.614	0.714
4–11	MDI	2	21.55	5.15	−16.40	1.02	17.43	0.191	0.987	0.801	0.816	0.589
4–11	MDSI	2	8.81	5.15	−3.67	1.35	5.02	0.430	0.985	0.737	0.947	0.708	0.806
12–17	MDI	1	21.00	30.20	9.20	11.32	2.12	0.899	0.857	0.625	0.970	0.878	0.803
12–17	MDI	2	21.00	7.32	−13.68	1.78	15.46	0.264	0.977	0.756	0.833	0.621
12–17	MDSI	2	9.99	7.32	−2.67	1.93	4.60	0.539	0.979	0.736	0.950	0.759	0.866
Model Two = SDQ Score (0 to 10) Predictor
Age Group	Variable	Cut Point¹	P/CAPA Estimate²	Modeled Estimate²	Bias²	FP³(Percent)	FN³(Percent)	Sens⁴	Spec⁴	PPV⁵	NPV⁵	AUCd⁶	AUCc⁶
4–11	MDI	3	21.55	25.07	3.52	12.37	8.85	0.589	0.842	0.507	0.882	0.716	0.764
4–11	MDSI	4	8.81	9.32	0.50	7.23	6.73	0.237	0.921	0.224	0.926	0.579	0.797
12–17	MDI	4	21.00	26.13	5.13	13.54	8.41	0.599	0.829	0.482	0.886	0.714	0.813
12–17	MDSI	5	9.99	11.04	1.05	5.41	4.35	0.564	0.940	0.510	0.951	0.752	0.862
Model Three=SDQ Score (0 to 10) and Impact Item (0 to 3) Predictors
Age Group	Variable	Cut Point⁷	P/CAPA Estimate²	Modeled Estimate²	Bias²	FP³(Percent)	FN³(Percent)	Sens⁴	Spec⁴	PPV⁵	NPV⁵	AUCd⁶	AUCc⁶
4–11	MDI		21.55	25.18	3.63	12.47	8.85	0.589	0.841	0.505	0.882	0.715	0.785
4–11	MDSI		8.81	7.97	−0.85	3.77	4.62	0.476	0.959	0.526	0.950	0.717	0.834
12–17	MDI		21.00	21.14	0.14	7.11	6.97	0.668	0.910	0.664	0.912	0.789	0.847
12–17	MDSI		9.99	7.93	−2.06	2.15	4.21	0.579	0.976	0.729	0.954	0.777	0.913
Model Four=Subset of up to 5 Individual SDQ Item Scores (0 to 2) and Impact Item (0 to 3) Predictors
Age Group	Variable	Cut Point⁸	P/CAPA Estimate²	Modeled Estimate²	Bias²	FP³(Percent)	FN³(Percent)	Sens⁴	Spec⁴	PPV⁵	NPV⁵	AUCd⁶	AUCc⁶
4–11	MDI⁹		21.55	21.12	−0.43	7.59	8.01	0.628	0.903	0.641	0.898	0.766	0.800
4–11	MDSI		8.81	8.92	0.11	4.77	4.66	0.471	0.948	0.466	0.949	0.710	0.873
12–17	MDI		21.00	21.24	0.24	7.84	7.60	0.638	0.901	0.631	0.904	0.770	0.851
12–17	MDSI		9.99	10.02	0.03	3.34	3.31	0.669	0.963	0.667	0.963	0.816	0.910

AUCc=area under ROC curve (continuous predictor); AUCd=area under ROC curve (dichotomous predictor); FN=false negative; FP=false positive; MDI=mental disorder with impairment; MDSI=mental disorder with severe impairment; NPV=negative predictive value; P/CAPA=preschool age psychiatric assessment (PAPA) or child and adolescent psychiatric assessment (CAPA); PPV=positive predictive value; ROC=receiver operating characteristic, Sens=sensitivity, Spec=specificity.

For models with a single predictor, a cut point of x indicates that a score ≥ x will result in a positive MDI or MDSI prediction; for models with multiple predictors, the cut point is not given (see Table 2, footnotes 7 and 8).

P/CAPA Estimate and Modeled Estimate refer to prevalence estimates of MDI or MDSI based on P/CAPA determinations and model predictions, respectively. Bias is the difference between modeled and P/CAPA estimates.

FP and FN are percentages of weighted false positive and false negative counts, respectively.

⁴

Sens (ie, sensitivity) is the weighted proportion of P/CAPA MDI or MDSI positive cases also predicted MDI or MDSI positive (ie, positive “hit” rate). Spec (ie, specificity) is the weighted proportion of P/CAPA MDI or MDSI negative cases also predicted MDI or MDSI negative (ie, 1 - “false alarm” rate).

⁵

PPV is the weighted proportion of predicted MDI or MDSI positive cases that are also P/CAPA MDI OR MDSI positive. NPV is the weighted proportion of predicted MDI or MDSI negative cases that are also P/CAPA MDI or MDSI negative.

⁶

AUCc is the concordance between the model (with continuous predictor) and P/CAPA MDI or MDSI, and AUCd is the concordance between the model (with predictor dichotomized by the cut point) and P/CAPA MDI or MDSI; AUCd = the average of sensitivity and specificity.

⁷

See Table 3 for cut points determined for combination of impact item score and SDQ score.

⁸

Cut point not given, because it would involve multiway tables or a detailed model description and cut point in terms of predicted probabilities.

⁹

Example formula: MDI (4-11 years) = if 1/(1 + exp[–(–3.7309 + 2.8261*Obed + 1.1503*Worry + 2.0109*Unhappy + 0.2049*Atten + 0.6565*Impact]) ≥ 0.46940 where Obed(iance), Worry, Unhappy, Adults, and Atten(tion) refer to the 5 items used in the abbreviated SDQ^§ and impact refers to a sixth SDQ item. Other formulas available upon request.