Table 2.
Psychometric criteria for evaluation of Vineland-3 and ABAS-3 use in CIBI outcome assessment.
| Criterion | Vineland-3 | ABAS-3 | |
|---|---|---|---|
| Structure | |||
| 1 | Factor structure | Inconsistent findings for subdomains across publications, no item-level analysis | Inconsistent findings, no item-level analysis |
| 2 | Measurement invariance/differential item or scale functioning | Inconsistent evidence for clinical IDD groups but good invariance evidence for sex, race/ethnicity, and SES | Invariance evidence for age and sex/gender |
| 3 | Measurement model guides scoring | Scoring based on theoretical, not empirical considerations | Scoring based on theoretical, not empirical considerations |
| Reliability | |||
| 4 | Scale reliability | Very strong for scored scales | Very strong for scored scales |
| 5 | Conditional reliability | Not evaluated | Not evaluated |
| 6 | Test–retest reproducibility | Very good | Very good |
| 7 | Test–retest stability | Not evaluated | Not evaluated |
| 8 | Inter-rater reliability | Good to excellent for raters within forms, across interviewers | Good to excellent for raters across forms with overlapping ages, good across adult self and adult informant reports |
| Validity | |||
| 9 | Content coverage | Good to excellent for broad SCI domain, weaker for specific SCI content areas | Good to excellent for broad SCI domain, weaker for specific SCI content areas |
| 10 | Construct coverage | Limited evidence to support domain sub-dimensions, no coverage of key SCI subdomains | Limited evidence to support domain sub-dimensions, no coverage of key SCI subdomains |
| 11 | Convergent validity | Strong evidence for broad domain scores across numerous studies, including criterion-related validity in ASD samples; small to medium correlations between Vineland domains/sub-domains and ASD instrument scores; correlations with ASD symptom dimensions are similar across communication, daily living, and social domains | Good evidence for broad domain scores across several studies, including criterion-related validity in ASD samples, correlations with ASD symptom measures are moderate and comparable in size across the three SCQ domains for SCQ; some evidence of convergent validity with ADOS-2 |
| 12 | Discriminant validity | Excellent | Excellent |
| 13 | Sensitivity to change | Evidence for significant and clinically meaningful gains across observational and randomized studies for socialization domain score | Not evaluated |
| Norming | |||
| 14 | Sample representativeness | Strong representativeness | Strong representativeness |
| 15 | Appropriate demographic adjustment | Yes, age adjustment for standard scores | Yes, age adjustment for standard scores |
| 16 | Traditional vs. continuous norming | Mixed norming, with accounting for distribution skewness and non-linear age trends in v-scale scoring | Traditional norming, but with a very large sample ~4000 and matching to US census |
| Scoring/Reporting | |||
| 17 | Validity indicators | Not present | Not present |
| 18 | Raw scores | Present by domain and subscale | Present by domain and skill area |
| 19 | Norm-referenced scores | Present by domain and subscale | Present by domain and skill area |
| Change Measurement | |||
| 20 | Growth scores | Present by domain | Not present |
| 21 | Reliable change scores | Not present | Not present |
| 22 | Clinically meaningful change | Published levels available | Not present |
Note. SCQ = Social Communication Questionnaire. ADOS-2 = Autism Diagnostic Observation Schedule-Second Edition.