Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 May 5.
Published in final edited form as: J Dev Behav Pediatr. 2010 Apr;31(3):181–191. doi: 10.1097/DBP.0b013e3181d5aa6d

Screening for Autism Spectrum Disorders in Children With Down Syndrome

Population Prevalence and Screening Test Characteristics

Carolyn DiGuiseppi *, Susan Hepburn †,, Jonathan M Davis *, Deborah J Fidler §, Sara Hartway , Nancy Raitano Lee , Lisa Miller *,, Margaret Ruttenber , Cordelia Robinson †,‡,**
PMCID: PMC4419691  NIHMSID: NIHMS678691  PMID: 20375732

Abstract

Objective

We assessed the prevalence of autism spectrum disorders (ASD) and screening test characteristics in children with Down syndrome.

Method

Eligible children born in a defined geographic area between January 1, 1996, and December 31, 2003, were recruited through a population-based birth defects registry and community outreach, then screened with the modified checklist for autism in toddlers or social communication questionnaire, as appropriate. Screen-positive children and a random sample of screen-negative children underwent developmental evaluation.

Results

We screened 123 children (27.8% of the birth cohort). Mean age was 73.4 months (range, 31–142). Compared to screen-negative children, screen-positive children had similar sociodemo-graphic characteristics but a lower mean developmental quotient (mean difference: 11.0; 95% confidence interval: 4.8–17.3). Weighted prevalences of autistic disorder and total ASD were 6.4% (95% confidence interval [CI]: 2.6%–11.6%) and 18.2% (95% CI: 9.7%–26.8%), respectively. The estimated minimum ASD prevalence, accounting for unscreened children, is 5.1% (95% CI: 3.3%–7.4%). ASD prevalence increased with greater cognitive impairment. Screening test sensitivity was 87.5% (95% CI: 66.6%–97.7%); specificity was 49.9% (95% CI: 37.0%–61.4%).

Conclusion

The prevalence of ASD among children with Down syndrome aged 2 to 11 years is substantially higher than in the general population. The modified checklist for autism in toddlers and social communication questionnaire were highly sensitive in children with Down syndrome but could result in many false positive tests if universal screening were implemented using current algorithms. Research needs include development of specific ASD screening algorithms and improved diagnostic discrimination in children with Down syndrome. Timely identification of these co-occurring diagnoses is essential so appropriate interventions can be provided.

Index terms: Down syndrome, Autism spectrum disorders, prevalence, test characteristics, Modified Checklist for Autism in Toddlers, Social Communication Questionnaire


The co-occurrence of autism spectrum disorders (ASD) and Down syndrome has important implications for intervention. Such children are likely to require different treatment and educational approaches than children with Down syndrome alone.1 However, the risk of ASD in children with Down syndrome has not been fully delineated. Large population-based studies of Down syndrome have reported autistic disorder in 1% to 2%, which was based on medical record reviews for psychiatric disorders.1 Rates of 3.3% to 11.0% have been reported in smaller clinical or convenience samples evaluated using clinical diagnostic criteria for autistic disorder.1 Two population-based studies have screened children with Down syndrome for ASD. Kent et al2 found ASD in 12.1% of 33 children who completed testing. They estimated population prevalences of ASD and autistic disorder to be 7% and 1.7%, respectively. Lowenthal et al3 reported ASD in 15.6% (autistic disorder 5.6%; Pervasive Developmental Disorder-Not Otherwise Specified 10.0%) of children with Down syndrome. However, neither study used state-of-the-art diagnostic tools.

Some investigators recommend a developmental approach to ASD diagnosis in cognitively impaired children: social or communication function must be both qualitatively different and more impaired than general cognitive function for an additional ASD diagnosis.4,5 Researchers taking this approach have generally reported lower ASD prevalences.1 Such estimates probably more accurately differentiate true ASD from characteristics of cognitive impairment. Epidemiological studies using a developmental approach to estimate the prevalence of autism in children with Down syndrome are lacking.

Epidemiological studies have typically identified at-risk subjects using standardized ASD screening tests. However, screening test characteristics may be affected by cognitive impairment or other conditions associated with Down syndrome. The social communication questionnaire performs well regardless of cognitive function in samples aged ≥4 years,69 although typically sensitivity is somewhat higher, and specificity somewhat lower, for children with cognitive impairment. Among cognitively impaired children, the modified checklist for autism in toddlers has demonstrated sensitivity of 70% to 88% (depending on criteria for failure), positive predictive value of 60% to 79%, and specificity of 38%.10-12 Evidence about the effect of demographic or clinical characteristics on performance of ASD screening instruments is limited. We found no studies that specifically examined the test characteristics of the social communication questionnaire or modified checklist for autism in toddlers, or clinical or demographic factors that might affect their performance, among children with Down syndrome.

This study aimed to assess the prevalence of ASD using comprehensive, rigorous diagnostic protocols, and the sensitivity and specificity of two autism screening tools in children with Down syndrome and to compare demographic and clinical characteristics among those who screened positive versus negative for ASD.

Methods

Study Design

This was a cross-sectional surveillance study of autism spectrum disorders (ASD) among children with Down syndrome.

Study Sample

Children with a chromosomal analysis documenting Down syndrome were eligible if born between January 1, 1996, and December 31, 2003, to a mother who was resident at delivery in 1 of 10 counties in north-central Colorado, currently alive, and residing with a parent or caregiver fluent in English or Spanish. There were 407,607 total live births in the catchment area during the study period.

Recruitment materials described a study of social, communication and behavioral needs in children with Down syndrome, without mentioning autism. We recruited participants through a statewide population-based registry of birth defects: “Colorado Responds to Children with Special Needs.” Colorado Responds to Children with Special Needs used vital statistics data to exclude deceased children then identify potentially eligible children based on birth date and maternal residence at delivery. Colorado Responds to Children with Special Needs staff members mailed recruitment letters to the most current registry address. Families were asked to authorize Colorado Responds to Children with Special Needs to release contact information or to contact researchers themselves. Whenever possible, undeliverable letters were re-sent to U.S. Postal Service forwarding addresses or new addresses identified by nonfee based searches of on-line and hard-copy directories. Most resent letters were also undeliverable.

Extensive community outreach was implemented. The Mile High Down Syndrome Association—a regional parent organization—advertised the study through newsletters and mailings to members. Families could respond either to researchers or to Mile High Down Syndrome Association. In the latter case, trained staff described the study and obtained authorization to release contact information to researchers. During workshops designed for parents of children with Down syndrome, conducted in collaboration with Mile High Down Syndrome Association or local school districts, researchers offered the opportunity to participate. Parents attending local symposia and other relevant events were recruited through exhibits and brochures.

Screening Procedure

A trained research assistant interviewed families by telephone (in person at workshops) to verify eligibility, explain the study, obtain verbal consent for screening, and screen in English or Spanish (as requested). A standard protocol was used, based on child's age and parent-reported verbal communication, to choose 1 of 2 brief screening instruments. The Social Communication Questionnaire (SCQ) (Current Form) was provided to parents of children aged ≥73 months and to children aged 60 to 72 months who regularly used phrases or sentences to communicate. The Modified Checklist for Autism in Toddlers (M-CHAT) was administered to parents of children aged <60 months and to children aged 60 to 72 months who communicated using only a few or single words because their developmental ages were assumed to fall within the age parameters of the instrument's development and validation samples.10,13

Modified Checklist for Autism in Toddlers (M-CHAT)13

The M-CHAT is a public domain screening tool in wide use because of its ease of administration, accessibility, and psychometric properties. The parent responds to 23 yes/no items. Failing any three items or any 2 of 6 critical items (focused on joint attention, social orienting, and imitation) indicates a positive screen. The rubric was established through discriminant function analysis on more than 1200 young children seen in clinics or referred for early intervention.13,14 Both scoring cutoffs resulted in sensitivity and specificity values above 0.90. Subsequent research supports the instrument's sensitivity and specificity among children functioning at a 2- to 3-year-old developmental level11 and those as old as 4 years.12 A “positive screen” was defined by meeting either scoring cutoff.

Social Communication Questionnaire (SCQ)15

The SCQ (Current Form) includes 40 “yes/no” items, derived from the autism diagnostic interview—revised,16 that focus on social reciprocity, communication disturbance, and repetitive behaviors often reported by parents of 4- to 5-year-old-children with an ASD. The SCQ strongly discriminates between ASD and non-ASD cases in both low- and high-risk samples, with sensitivity and specificity ranging from 0.85 to 0.88 and from 0.72 to 0.75, respectively.68 Somewhat lower sensitivity (0.68) has been reported for children younger than 5 years.17 A score of 15 was used as the cutoff, as suggested by the developers.

Children whose score exceeded the relevant cut-off for the screening test, and a random sample of two thirds of children whose scores did not exceed the cutoff, were invited for a clinical evaluation of social, communication, and behavioral skills. For screen-negative child not selected for evaluation, the parent was informed that the child was demonstrating no risk for significant social and communication difficulties. Age-appropriate recommendations relevant to any parental concerns were subsequently mailed to the parent.

Evaluation Measures

Autism Diagnostic Observation Schedule—Generic (ADOS-G)18

The Autism Diagnostic Observation Schedule-Generic (ADOS-G) uses developmentally appropriate social and toy-based interactions in a 45 to 60 minute semi-structured standardized play interview to elicit symptoms of autism. There are four different modules, each directed at a particular level of language ability. Given our participants' language/developmental levels, we administered only Modules 1 and 2. The ADOS-G classifies children as “Autism,” “Autism Spectrum,” or “Not Autism” based on exceeding (or not) cutoffs on total score and two symptom domains. The ADOS has been psychometrically validated across a wide range of ages and severity levels in autism.18,19 The ADOS-G was administered to all clinically evaluated children regardless of mental age (MA), although at the start of this study, available ADOS modules were not recommended for children with MA <18 months.18

Autism Diagnostic Interview-Revised (ADI-R)16,20

The Autism Diagnostic Interview-Revised (ADI-R) is a structured, standardized parent interview, with more than 100 questions, that assesses the presence and severity of autism symptoms in early childhood in social relatedness, communication, and repetitive and/or restrictive behaviors. This instrument yields an algorithm score and cutoffs for a diagnostic classification of autism but does not differentiate broader spectrum disorders. The ADI-R classification differentiates autism from other developmental disorders with sensitivity and specificity over 0.90, respectively, for subjects with MA ≥18 months. The ADI-R was administered to all children in the study regardless of MA.

Developmental Assessment

Participants with an estimated developmental level at or below 5 years were provided the Mullen Scales of Early Learning.21 This measure provides a developmental assessment for children aged 1 to 68 months and estimates verbal and nonverbal problem-solving abilities. Participants with an estimated developmental level above 5 years were assessed with the Differential Ability Scales,22 which contains cognitive and achievement batteries for children aged 2.5 to 17 years. The Differential Ability Scales also estimates verbal and nonverbal abilities. Both instruments provide age-based standard scores and age-equivalent scores for each subtest.

To avoid floor effects due to significant cognitive impairment, test selection was based on the participant's developmental level rather than chronological age (for example, an 8 year old with the MA of a 1 year old received the Mullen Scales of Early Learning). Thus, norm-referenced, age-based standard scores could not be calculated for all participants. To create a consistent developmental measure for all participants, developmental quotients were calculated by dividing overall, verbal and non-verbal age-equivalent scores (i.e., MA) by chronological age, consistent with previous studies of young children with autism.23,24 Developmental quotient scores were chosen because age-equivalent scores do not capture how much a child's skill level deviates from chronological age (e.g., they do not differentiate between an 8 and 4 year old who both function at a 12-month developmental level). Developmental quotients were categorized into mild (50.00-<70.00), moderate (40.00-<50.00), severe (25.00 to <40.00), or profound (<25.00) level of impairment.

Vineland Adaptive Behavior Scales—Second Edition (Vineland II)25

The Vineland-II Survey Interview Form was administered to parents to estimate each child's adaptive functioning in social, communication, daily living, and motor skills.25 The Vineland yields standard scores and age equivalents. Children with autism may demonstrate a unique profile of strengths and weaknesses on the Vine-land Adaptive Behavior Scales.26

Demographic and Medical Data

Demographic information was obtained from birth certificates and by questionnaire during the clinic visit. History of medical and sensory conditions was collected by questionnaire during the clinic visit or by mail afterward.

Evaluation Procedures

One or both parents attended 2 to 3 2-hour evaluation appointments. Parents completed checklists before the first visit. Written consent was obtained at the first clinical visit. Evaluations were conducted at JFK Partners, University of Colorado Denver, or at Colorado State University, by clinicians with extensive experience with children with developmental disabilities. All project clinicians established and maintained research reliability on the ADOS and ADI-R. All sessions were videotaped (with permission); 40% of evaluations were reviewed to assess reliability of administration and scoring of diagnostic tools. For the ADOS, ADI-R, and Vineland adaptive behavior scales, inter-observer reliability remained at or above 85%.

Diagnostic Status

Final diagnostic status was based on an overall clinical diagnosis driven by expert clinical opinion, given all information obtained in the evaluation, including both ADOS and ADI classification. Potential problems using these instruments in children with MA below 18 months were considered. The social and communicative profile was carefully weighed in the context of overall developmental functioning. Based on DSM-IV TR criteria, children were classified into (1) Autistic Disorder, (2) Pervasive Developmental Disorder-Not Otherwise Specified, or (3) unaffected (no ASD). We defined ASD to include children with either autistic disorder or Pervasive Developmental Disorder-Not Otherwise Specified. Reliability of clinical impressions was ascertained by two to three psychologists (within a team of four, always including S.H.) reviewing videotapes and charts for 31 cases (40%) and independently determining a clinical diagnosis. Kappas between diagnostician pairs ranged from 0.67 to 0.88. Consensus discussions among clinicians were implemented routinely for the first 14 cases to develop and maintain consistency in conceptualization. When disagreements on diagnostic status occurred in subsequent cases (autistic disorder vs Pervasive Developmental Disorder-Not Otherwise Specified, n = 2; Pervasive Developmental Disorder-Not Otherwise Specified vs no ASD, n = 4), consensus was reached through in-depth case review and discussion with the psychologist who conducted the evaluation as primary diagnostician. In determining final diagnostic status, the ADOS and cognitive testing were weighted more than the ADI-R, particularly when the child's MA was <18 months.

Feedback to Families

After the evaluation, each family received a written report with findings and treatment recommendations and was offered a feedback session with the psychologist. Forty-eight families attended a feedback session.

Data Analysis

We compared socio-demographic characteristics of screened and unscreened children with Down syndrome in the total birth cohort, and of screen-negative versus-positive subjects, using chi-square, Fisher's exact, and Student's t-tests as appropriate.

ASD prevalence rates were calculated for all enrolled children and demographic and cognitive subgroups. Data from screen-negative subjects were weighted (by inverse selection probability) to account for the sampling fraction for selection. Weighted associations and means with corresponding 95% confidence intervals were determined, using Taylor series approximations to determine variance in the complex sampling design.

Two sensitivity analyses examined the potential effect on prevalence estimates if parents with concerns about their child's social and communication development preferentially enrolled. First, we used as the denominator all non-deceased children with Down syndrome born in the catchment area during the study period, after excluding families whose mailing was undeliverable. Second, we estimated the lowest possible prevalence of ASD among children with Down syndrome in the geographic catchment, using as the denominator all non-deceased children with Down syndrome in the birth cohort.

Sensitivity, specificity, and corresponding 95% confidence intervals were determined for the modified checklist for autism in toddlers and SCQ using final clinical diagnosis as the gold standard.

Univariate logistic regression was used to evaluate whether child health or sensory problems increased the likelihood of a false positive result on either screening test, among children who were clinically evaluated and found not to have an ASD. Odds ratios with 95% confidence intervals were calculated, comparing medical and sensory conditions of screen-positive versus screen-negative children in this subset.

Human Subjects Protection

The Colorado Multiple Institutional Review Board and the Institutional Review Board of the Colorado Department of Public Health and Environment approved this study.

Results

Figure 1 shows recruitment among the 498 potentially eligible children with Down syndrome. We screened 123 children: 40.4% of 304 children whose families were presumed to have received our letter, and 27.8% of all 443 potentially eligible non-decreased children. Enrolled children were significantly younger and more likely to be male and non-Hispanic white than all other non-deceased children with Down syndrome in the birth cohort (Table 1). Compared to mothers of non-enrolled children, mothers of enrolled subjects were on average older and had more education.

Figure 1.

Figure 1

Recruitment and enrollment of children with Down syndrome born to mothers resident in the study catchment area, 1996 to 2003.

Table 1. Characteristics of Enrolled Versus Not Enrolled Children With Down Syndrome Born to Mothers Resident in the Study Catchment Area, 1996–2003.

Characteristicsa Enrolled, N (%) Not Enrolled, n (%) p
Child's gender .039
 Male 80 (65.0) 173 (54.2)
 Female 43 (35.0) 146 (45.8)
Child's race/ethnicity <.0001
 White, non-Hispanic 101 (82.1) 163 (51.1)
 White, Hispanic 15 (12.2) 107 (33.5)
 Black 6 (4.9) 29 (9.1)
 Native American 0 (0.0) 5 (1.6)
 Asian/Pacific Islander 1 (0.8) 15 (4.7)
Child's birth year .035
 1996 10 (8.1) 33 (10.3)
 1997 5 (4.1) 37 (11.6)
 1998 11 (8.9) 27 (8.5)
 1999 10 (8.1) 39 (12.2)
 2000 17 (13.8) 42 (13.2)
 2001 14 (11.4) 41 (12.9)
 2002 33 (26.8) 46 (14.4)
 2003 23 (18.7) 54 (16.9)
n (Mean) n (Mean)

Maternal ageb 123 (33.9) 320 (31.2) .0001
Maternal educationb 122 (14.8) 317 (12.8) <.0001
a

From the birth certificate.

b

On child's birth date.

Screen-positive and screen-negative children did not differ significantly in demographic characteristics, primary language, or screening test administered (Table 2). Most screen-positive and (selected) screen-negative children completed a clinical evaluation (Fig. 1). Of those not evaluated, one family refused and the rest failed to make or keep appointments. Among screen-negative children selected for evaluation, screening test results did not differ between those who did and did not complete the clinical evaluation, either for the Social Communication Questionnaire (mean = 6.5 vs 8.3, p = .220) or the modified checklist for autism in toddlers (mean number of positive items = 21.8 vs 21.7, p = .923).

Table 2. Characteristics of 123 Children With Down Syndrome Aged 2 to 11 Years Who Were Enrolled and Screened for Autism Spectrum Disorders.

Characteristic Total Sample Screen Negative Screen Positive p



right SD Mean SD Mean SD
Chronological age at screening (mo) (range) 73.4 (31–142) 28.0 76.3 (33–142) 29.4 69.7 (31–120) 25.5 .20
N % N % N %

Gender .95
 Male 80 65.0 46 64.8 34 65.4
 Female 43 35.0 25 35.2 18 34.6
Racea .54
 White 116 94.3 67 94.3 49 94.2
 Black 6 4.9 3 4.2 3 5.8
 Asian/Pacific Islander 1 0.8 1 1.4 0 0.0
Ethnicitya .17
 Not Hispanic 108 87.8 63 88.7 45 86.5
 Hispanic 15 12.2 8 11.3 7 13.5
Primary language .07
 English 115 93.5 69 97.2 46 88.5
 Spanish 8 6.5 2 2.8 6 11.5
Birth year .15
 1996 10 8.1 8 11.3 2 3.9
 1997 5 4.1 4 5.6 1 2.0
 1998 11 8.9 5 7.0 6 11.5
 1999 10 8.1 8 11.3 2 3.9
 2000 17 13.8 8 11.3 9 17.3
 2001 14 11.4 6 8.5 8 15.4
 2002 33 26.8 22 31.0 11 21.2
 2003 23 18.7 10 14.1 13 25.0
Test administered .23
 M-CHAT 85 69.1 46 64.8 39 75.0
 SCQ 38 30.9 25 35.2 13 25.0

M-CHAT, Modified checklist for autism in toddlers; SCQ, social communication questionnaire.

a

From birth certificate.

Among clinically evaluated children, screen-positive children were significantly more likely to have married parents than screen-negative children (97.0 vs 78.6%, p = .01). Clinically evaluated screen-positive and screen-negative children otherwise had similar family socioeconomic status and maternal and paternal age, education, and hours of outside employment (data not shown).

Prevalence of Autism Spectrum Disorders (ASD)

The prevalence of autistic disorder, pervasive developmental disorder-not otherwise specified (PDD-NOS), and all ASD combined among clinically diagnosed children with Down syndrome is shown in Table 3.

Table 3. Prevalence of Autism Spectrum Disorders Among Children With Down Syndrome Aged 2 to 11 Years, Overall and By Demographic and Cognitive Characteristics.

Characteristic Autistic Disorder Weighted Percent (95% CI) Pervasive Developmental Disorder-Not Otherwise Specified Weighted Percent (95% CI) Total Autism Spectrum Disorder Weighted Percent (95% CI)
Total 6.4 (2.6–11.6) 11.8 (7.0–18.4) 18.2 (9.7–26.8)
Gender
 Male 6.8 (0.2–13.5) 17.1 (9.9–26.1) 23.9 (11.9–35.9)
 Female 5.8 (1.5–15.8) 2.9 (0.0–8.6) 8.7 (0.0–18.4)
Race
 White Non-Hispanic 6.8 (0.8–12.7) 12.9 (0.0–21.1) 19.7 (9.9–29.4)
 All other races 5.5 (0.0–16.4) 8.4 (0.0–24.3) 13.9 (0.0–32.4)
Ethnicity
 Hispanic 0.0 (0.0) 0.0 (0.0) 0.0 (0.0)
 Not Hispanic 7.5 (1.5–13.5) 13.8 (5.5–22.2) 21.3 (11.5–31.2)
Birth year
 1996–1999 9.5 (3.1–22.5) 9.5 (3.1–22.5) 19.0 (1.3–36.7)
 2000–2001 3.7 (0.0–11.0) 14.9 (0.0–30.8) 18.5 (1.5–35.5)
 2002 7.5 (0.0–17.9) 7.5 (0.0–17.9) 15.0 (0.1–29.3)
 2003 5.4 (1.1–22.0) 16.2 (0.0–33.6) 21.6 (1.2–41.2)
Severity of cognitive impairmenta
 Mild 0.0 (0.0–10.9) 9.1 (0.0–19.4) 9.1 (0.0–19.4)
 Moderate 0.0 (0.0–20.6) 19.9 (1.4–38.3) 19.9 (1.4–38.3)
 Severe 27.8 (7.8–47.8) 16.2 (0.0–33.8) 44.1 (21.1–67.2)
a

Missing data for 12 children who did not complete cognitive testing.

Sensitivity Analyses

If one assumes that we identified all children with ASD among the 304 families who could be contacted by mail, the estimated prevalence of ASD among children with Down syndrome is 7.4% (95% confidence interval [CI]: 4.8–10.7%). If one assumes that we identified all children with ASD within the entire cohort of 443 non-deceased children, the estimated minimum prevalence of ASD among children with Down syndrome is 5.1% (95% CI: 3.3–7.4%).

Prevalence of ASD by Demographic and Clinical Characteristics

The prevalence of PDD-NOS was significantly higher in males, and no Hispanic children had an ASD (Table 3). Prevalence did not vary significantly by birth year or race.

Cognitive testing was completed on 64 children. Autistic disorder was diagnosed in 27.8% (95% CI: 7.8%-47.8%) of children with severe cognitive impairment but in no children with mild or moderate impairment (Table 3). PDD-NOS was diagnosed about twice as often in children with moderate or severe impairment as in children with mild impairment.

Among cognitively tested children, 11 (17.2%) had a mental age <18 months. After excluding these children, the overall weighted prevalence of ASD was 130% (95% CI 4.0-22.0%), and all cases were diagnosed with PDD-NOS. Within this subgroup, the prevalence of PDD-NOS was 93% (95% CI: 0.0%-20.0%) among children with mild cognitive impairment, 19.9% (95% CI: 1.4%-38.5%) with moderate impairment, and 13.1% (95% CI 0.0%-37.5%) with severe impairment.

Developmental Characteristics of Screen-Positive Versus Screen-Negative Children

Significantly more screen-positive than screen-negative children were diagnosed with autistic disorder or PDD-NOS (Table 4). Screen-positive children had more severe cognitive impairment and a lower mean developmental quotient, with a mean difference in developmental quotient between groups of 11.0 (95% CI: 4.8-17.3). Scores on the Vineland Adaptive Behavior Scales did not differ between screen-positive and screen-negative children; however, parents of only about 75% of clinically evaluated children completed the Vineland Adaptive Behavior Scales.

Table 4. Developmental Characteristics of Children With Down Syndrome Aged 2–11 Years Who Were Enrolled and Screened for Autism Spectrum Disorders.

Characteristic Screen-Positive Weighted Percent (95% CI) Screen-Negative Weighted Percent (95% CI) p
Autism spectrum disorder 35.6 (23.2–50.2) 6.4 (1.9–20.2) .018
Diagnosis <.001
 Autistic disorder 13.3 (6.3–26.3) 0.0 (0.0–10.6)
 Pervasive developmental disorder—not otherwise specified 22.2 (12.6–36.4) 6.4 (1.9–20.2)
Severity of cognitive impairmenta .001
 Mild 28.5 (13.2–43.9) 63.3 (45.6–81.0)
 Moderate 31.4 (15.6–47.2) 20.0 (0.05–34.7)
 Severe 22.9 (8.6–37.1) 16.7 (3.0–30.3)
 Profound 17.1 (4.3–20.0) 0.0 (0.0–0.0)
Standardized testing
 Developmental quotienta 41.0 (36.1–45.9) 52.1 (48.2–56.0) .001
 Vineland communication standard scoreb 45.2 (37.6–52.8) 43.8 (35.8–51.9) .80
 Vineland socialization standard scoreb 48.2 (40.3–56.1) 45.5 (37.2–53.8) .63
 Vineland daily living standard scoreb 46.6 (39.1–54.2) 42.9 (33.9–51.8) .51
 Vineland motor skills standard scorec 67.7 (59.8–75.7) 74.0 (68.8–79.2) .20
a

Missing data for 2 screen-negative (SN) and 10 screen-positive (SP) children.

b

Mean = 100; missing data for 6 SN and 12 SP children.

c

Mean = 100; missing data for 10 SN and 10 SP children.

Screening Test Performance

The combined sensitivity of the two screening tests for identification of any ASD was 87.5% (95% CI: 66.6-97.7%); specificity was 49.9% (95% CI: 37.0-61.4%). Sensitivity and specificity of the Social Communication Questionnaire were 100.0% (95% CI: 60.7-100.0%) and 57.1% (95% CI: 32.8-76.9%), respectively. For the Modified Checklist for Autism in Toddlers, sensitivity was 81.8% (95% CI: 55.0-96.4%) and specificity 46.8% (95% CI: 33.2-60.7%).

Thirty-four children who did not have an ASD completed a detailed medical history. Among these children, a (falsely) positive screening test was significantly more likely if the child had a known hearing problem (odds ratios = 8.4; 95% CI: 1.7-42.4; p = .01) or a persistent vision problem despite wearing glasses (odds ratios = 24.0; 95% CI: 1.1-505.2; p = .04). Children born prematurely were nearly four times as likely to have a false positive screen (odds ratios = 39; 95% CI: 0.8-18.2), a difference of borderline significance (p = .08). There was no association between a history of heart disease, seizures or ear infections and a false positive screening test.

Discussion

In a geographically based sample of children with Down syndrome aged 2 to 11 years of age who were evaluated using comprehensive diagnostic protocols and rigorous testing procedures, the estimated prevalence of autism spectrum disorder (ASD) was 18.2% and that of autistic disorder (AD) was 6.4%. Our estimate for ASD among children with Down syndrome is 17 to 20 times higher than the estimated ASD prevalence in the general population.27,28 However, similar or higher rates of AD have been reported in other genetic disorders likewise characterized by severe cognitive impairment, such as Fragile X and tuberous sclerosis.29

Biological bases for co-occurring ASD and Down syndrome have been proposed. The centrosome has been implicated in numerous cellular processes that lead to various neurological diseases associated with brain structural abnormalities, through neuronal migration pathways30 and microtubular organization.31 These processes could be etiologically related to ASD, as region-specific and unbalanced early brain overgrowth has been identified in children with ASD.32 Molloy et al33 through linkage analysis, identified a 2.7 Mb region on 21q among 34 autism-affected relative pairs selected for language regression. This region includes genes with known or possible roles in cellular differentiation, apoptosis, virus infection susceptibility, and responses to oxidation-reduction changes, all of which have been potentially implicated in autism.33 However, in a different ASD population, no linkage to chromosome 21 was identified.34 The role of joint biological mechanisms in these two disorders requires further investigation.

Our prevalence estimates are somewhat higher than the prevalence rates of 3% and 5.6% for AD, and 9.1% and 10% for Pervasive Developmental Disorder-Not Otherwise Specified, respectively, reported in two previous population-based studies.2,3 Neither study evaluated screen-negative children, who accounted for 20% of our ASD cases. Kent et al2 included younger ages, when symptoms suggestive of ASD may not yet manifest. Our expanded recruitment methods may also have increased our estimate. We sent letters to families identified from a statewide birth defects registry, who accounted for 31% of our participants. Use of population-based registries may increase recruitment of low income, less educated, and non-English speaking families, who may not belong to local parent groups or attend workshops and may have reduced access to educational testing, school services, and specialized medical care.

The quality of our diagnostic evaluation process supports the validity of our prevalence estimates. Neither Kent et al2 nor Lowenthal et al3 used comprehensive diagnostic protocols to establish the diagnosis. Lowenthal et al3 based their estimate solely on the parent-reported Social Communication Questionnaire,15 with no subsequent clinical evaluation. Kent et al2 administered the As-pergers Syndrome Screening Questionnaire and the Childhood Autism Rating Scale, with screen-positive children then observed at school and interviewed and observed by a principal speech therapist. Substantial clinical experience is essential in diagnosing ASD within the context of known cognitive impairment.35

When this study was initiated, available Autism diagnostic observation schedule modules were not recommended for children with mental ages below 18 months.18 Therefore, the diagnostic measures we used may not have been developmentally appropriate for some children. We considered this issue in the diagnostic conceptualization process, which was driven by expert clinical opinion and included careful weighing of the child's social and communicative profile against the background of overall developmental functioning. As measures of autistic symptoms for cognitively impaired persons improve, their specificity within this population may also improve.

We used a DSM-IV TR symptom checklist to diagnose each child, specifying information sources for each symptom from both the Autism diagnostic observation schedule and Autism diagnostic interview-revised. Project clinicians were confident that children identified with AD, all of whom had mental ages <18 months, demonstrated social and communicative impairments consistent with “true” co-occurring autism. Clinicians were less confident about identifications of children with Pervasive Developmental Disorder-Not Otherwise Specified. These children usually presented with significant problems in communication and repetitive behavior; whereas their social style demonstrated less reciprocity than expected for their overall developmental level, core social relatedness was not as impaired. The question remains whether these children truly had an ASD, or whether cognitive, temperament, attention, and motor factors combined to influence reciprocity and communicative development.

Severely impaired children had a substantially higher prevalence of ASD than mildly impaired children in our study, consistent with existing literature.36,37 Studies examining children with intellectual disability have reported overall prevalence rates ranging from 9% to 17% for AD and 20% or more for ASD.38 Molloy et al39 suggested that the observed association between AD and cognitive impairment might be explained by a common neural mechanism for both disorders, specifically, neural network disconnectivity. The relationship between ASD and cognitive impairment cannot be wholly explained by performance problems with the Autism Diagnostic Interview-Revised and Autism Diagnostic Observation Schedule-Generic among cognitively impaired children. In a study of 184 subjects aged 5 to 20 years old with intellectual disability, the Autism diagnostic interview-revised and Autism diagnostic observation schedule performed well for diagnosing ASD relative to clinical examinations by two experienced clinicians.36 Another study reported that children aged 4 to 16 years with both Down syndrome and autism were significantly more impaired in cognition, language abilities, and adaptive behavior skills than those with Down syndrome alone; however, the deficits in social interaction and communication and increased repetitive and restricted behaviors that characterized children with dual diagnoses were not entirely explained by their more severe cognitive impairment.39

Compared to clinical diagnosis as the “gold standard,” both screening tests performed well in identifying cases of ASD, with a sensitivity of 88%. However, specificity was only 50%, for which there are several possible explanations. First, children with developmental disability often demonstrate delays in social and communicative development, which the screening instruments may capture. Screen-positive children were on average significantly more cognitively impaired than screen-negative children. This most likely reflects true differences in the prevalence of ASD in relation to cognitive function, but also that children with significant cognitive impairment are likely to be impaired socially and communicatively as well. Second, children with Down syndrome are at increased risk for sensory conditions (e.g., hearing loss) and motor difficulties (e.g., hypotonia), which may affect the timing and fluidity of their social and communicative behaviors in a manner detected by screening tools, but qualitatively different from the core social relatedness problems of autism. False positive screening results were more likely in children with hearing or persistent vision problems. Efforts to correct sensory deficits before screening may be appropriate. Finally, recent studies suggest that children with Down syndrome demonstrate clinically significant executive function deficits (i.e., planning, shifting attention, perseveration, cognitive inflexibility),40 which affect social and communicative functioning, but in a way distinct from reciprocity problems associated with autism. Many items on the screening tools tap aspects of executive function. Children with inflexible behavioral styles or difficulty coordinating multiple behaviors simultaneously may screen positive for autism. Further evaluation with an experienced clinician is necessary to disentangle executive dysfunction from poor social relatedness. Future work should be directed towards identifying appropriate screening algorithms for children with Down syndrome to decrease false positives. Meanwhile, clinicians should supplement screening questionnaires with direct observation, attending to social orienting, communicative intention, emotional contagion, and other aspects of core social relatedness that differentiate autism from global developmental delay.

The low specificity among children with Down syndrome may be of less concern from a population perspective since, given the low prevalence of Down syndrome, the number of children with false positive results will be small. Although high false positive rates have important implications for resources and family burden, sacrificing some specificity to maintain a high sensitivity may be appropriate given the importance of identifying the dual diagnosis for intervention and treatment purposes.

Strengths and Limitations

Assessment of a sample of screen-negative children allowed us to more accurately estimate the test characteristics of the Social Communication Questionnaire and Modified Checklist for Autism in Toddlers and the estimated prevalence in the population, because some affected children were missed by the screening test. Screen-negative children selected for evaluation were more likely to refuse or drop out than were screen-positive children (32 vs. 13%, respectively). The higher drop out rate among screen-negative children is unlikely to have biased our results, however, as screening test results suggested similar levels of clinical symptoms among screen-negative children who completed the evaluation and those who did not. Families dropped out most often due to the time demands of the study. A less intensive evaluation process might have resulted in less attrition, but this was not feasible because multiple sessions and measures were required to evaluate the potential dual diagnosis.

We attempted to recruit a geographically based birth cohort of children with Down syndrome to obtain a population-based prevalence estimate. A population-based birth defects registry allowed us to identify children from families who, due to cultural or language barriers or reduced access to testing and services, may not be identified in clinical or educational samples. However, despite extensive recruitment efforts involving both public health and community partners, we were able to screen only 28% of the children with Down syndrome estimated to have been born in the region during the study period. Nearly one-third of families in the Colorado Responds to Children with Special Needs birth defects registry could not be contacted at all due to missing or out-of-date addresses. Migration out of the geographic area impeded both contact and clinic visits. Reluctance to participate may have stemmed from busy family schedules and the sense that their children had been tested enough.

The smaller than anticipated sample size reduced study power to identify differences between groups in demographic, developmental, and clinical characteristics. In addition, there were significant differences between children who did and did not enroll in gender, race/ethnicity, and maternal age, all of which are or may be positively associated with ASD.41 For the reason that enrolled participants may therefore have been more likely to have AD, we may have overestimated the prevalence of co-occurring disorders. Further, our results are likely to be most generalizable to white, non-Hispanic male children with Down syndrome. Our estimated minimum comorbid rate of 7%, which assumes that we enrolled all children with co-occurring Down syndrome and ASD among contacted families in the birth cohort, is identical to that of Kent et al2 and may be a more accurate estimate.

Conclusions

The study results suggest that the prevalence of autism spectrum disorder among children with Down syndrome aged 2 to 11 years is substantially higher than in the general population. Two standardized screening tests, the Modified Checklist for Autism in Toddlers and the Social Communication Questionnaire, were highly sensitive in identifying children with Down syndrome and autism spectrum disorder. However, many false positive tests, which have important resource implications, could result if universal screening were implemented in this population using current algorithms. Assuring that sensory deficits are corrected before screening may help avoid false positive tests. Efforts to analyze items on the Modified Checklist for Autism in Toddlers and develop an algorithm specifically for children with Down syndrome are currently underway, which may reduce false positive tests. Meanwhile, education of clinicians on the potential importance of parent reports of symptoms suggestive of autism spectrum disorder, and facilitation of early screening and diagnosis of autism spectrum disorder in this population, are essential, as is recognition of the need to place parent reports, screening test results and clinical observations within a developmental context. Access to screening for older children might be improved by including a screening measure in the assessment process for the Individualized Family Service Plan or Individualized Education Plan, or by working with service providers to implement screening. Identification of children with co-occurring diagnoses allows them to obtain appropriate therapeutic and educational interventions. Their families might also feel more supported in the dual diagnosis if the individual differences in children with Down syndrome were recognized consistently from an early age (i.e., “some children with this genetic disorder are very social, others are not as social”).

Acknowledgments

We gratefully acknowledge Russel Rickard, Amy Alman, and Carol Stanton, Colorado Responds to Children with Special Needs, who assisted with recruitment and data analysis, Laurie Herrera, Mile High Down Syndrome Association, for assistance with recruitment and enrollment, and Audrey Blakely-Smith, PhD, and Amy Philofsky, PhD for assistance with the clinical quality assurance.

This investigation was supported by the National Center on Birth Defects and Developmental Disabilities, Centers for Disease Control and Prevention, through Cooperative Agreement RTOI2005-1/2-416 with the Association of University Centers on Disabilities; University Center of Excellence in Developmental Disabilities Education, Research, and Service Grant 90DD0632 from the Administration on Developmental Disabilities, Administration for Children and Families; and Leadership Education in Neurodevelopmental Disabilities Grant 5-T73-MC11044-02-00 from the Maternal and Child Health Bureau, Health Resources and Services Administration.

References

  • 1.Reilly C. Autism spectrum disorders in Down syndrome: a review. Res Autism Spectr Disord. 2009;3:829–839. [Google Scholar]
  • 2.Kent L, Evans J, Paul M, Sharp M. Comorbidity of autistic spectrum disorders in children with Down syndrome. Dev Med Child Neurol. 1999;41:153–158. doi: 10.1017/s001216229900033x. [DOI] [PubMed] [Google Scholar]
  • 3.Lowenthal R, Paula CS, Schwartzman JS, Brunoni D, Mercadante MT. Prevalence of pervasive developmental disorder in Down's syndrome. J Autism Dev Disord. 2007;37:1394–1395. doi: 10.1007/s10803-007-0374-4. [DOI] [PubMed] [Google Scholar]
  • 4.Filipek PA, Accardo PJ, Ashwal S, et al. Practice parameter: screening and diagnosis of autism: report of the Quality Standards Subcommittee of the American Academy of Neurology and the Child Neurology Society. Neurology. 2000;55:468–479. doi: 10.1212/wnl.55.4.468. [DOI] [PubMed] [Google Scholar]
  • 5.Ozonoff S, Goodlin-Jones BL, Solomon M. Evidence-based assessment of autism spectrum disorders in children and adolescents. J Clin Child Adolesc Psychol. 2005;34:523–540. doi: 10.1207/s15374424jccp3403_8. [DOI] [PubMed] [Google Scholar]
  • 6.Berument S, Rutter M, Lord C, Pickles A, Bailey A. Autism screening questionnaire: diagnostic validity. Br J Psychiatry. 1999;175:444–451. doi: 10.1192/bjp.175.5.444. [DOI] [PubMed] [Google Scholar]
  • 7.Chandler S, Charman T, Baird G, et al. Validation of the Social Communication Questionnaire in a population cohort of children with autism spectrum disorders. J Am Acad Child Adolesc Psychiatry. 2007;46:1324–1332. doi: 10.1097/chi.0b013e31812f7d8d. [DOI] [PubMed] [Google Scholar]
  • 8.Charman T, Baird G, Simonoff E, et al. Efficacy of three screening instruments in the identification of autistic-spectrum disorders. Br J Psychiatry. 2007;191:554–559. doi: 10.1192/bjp.bp.107.040196. [DOI] [PubMed] [Google Scholar]
  • 9.Witwer AN, Lecavalier L. Autism screening tools: an evaluation of the social communication questionnaire and the developmental behaviour checklist-autism screening algorithm. J Intellect Dev Disabil. 2007;32:179–187. doi: 10.1080/13668250701604776. [DOI] [PubMed] [Google Scholar]
  • 10.Kleinman JM, Robins DL, Ventola PE, et al. The Modified Checklist for Autism in Toddlers: a follow-up study investigation the early detection of autism spectrum disorders. J Autism Dev Disord. 2008;38:827–839. doi: 10.1007/s10803-007-0450-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Pandey J, Berbalis A, Robins DL, et al. Screening for autism in older and younger toddlers with the modified checklist for autism in toddlers. Autism. 2008;12:513–535. doi: 10.1177/1362361308094503. [DOI] [PubMed] [Google Scholar]
  • 12.Snow AV, Lecavalier L. Sensitivity and specificity of the modified checklist for autism in toddlers and the social communication questionnaire in preschoolers suspected of having pervasive developmental disorders. Autism. 2008;12:627–644. doi: 10.1177/1362361308097116. [DOI] [PubMed] [Google Scholar]
  • 13.Robins DL, Fein D, Barton ML, Green JA. The modified Checklist for Autism in Toddlers: an initial study investigating the early detection of autism and pervasive developmental disorders. J Autism Dev Disord. 2001;31:131–144. doi: 10.1023/a:1010738829569. [DOI] [PubMed] [Google Scholar]
  • 14.Robins DL, Dumont-Mathieu TM. Early screening for autism spectrum disorders: update on the modified checklist for autism in toddlers and other measures. J Dev Behav Pediatr. 2006;27:111–119. doi: 10.1097/00004703-200604002-00009. [DOI] [PubMed] [Google Scholar]
  • 15.Rutter M, Bailey A, Lord C. Social Communication Questionnaire (SCQ) Los Angeles, CA: Western Psychological Services; 2003. [Google Scholar]
  • 16.Lord C, Rutter M, LeCouteur A. Autism diagnostic interview revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord. 1994;24:659–685. doi: 10.1007/BF02172145. [DOI] [PubMed] [Google Scholar]
  • 17.Corsello C, Hus V, Pickles A, et al. Between a ROC and a hard place: decision making and making decisions about using the SCQ. J Child Psychol Psychiatry. 2007;48:932–940. doi: 10.1111/j.1469-7610.2007.01762.x. [DOI] [PubMed] [Google Scholar]
  • 18.Lord C, Rutter M, DiLavore P, et al. Autism Diagnostic Observation Schedule–WPS Edition. Los Angeles, CA: Western Psychological Services; 1999. [Google Scholar]
  • 19.Risi S, Lord C, Gotham K, et al. Combining information from multiples sources in the diagnosis of autism spectrum disorders. J Am Acad Child Adolesc Psychiatry. 2006;45:1094–1103. doi: 10.1097/01.chi.0000227880.42780.0e. [DOI] [PubMed] [Google Scholar]
  • 20.Rutter M, LeCouteur A, Lord C. Autism Diagnostic Interview-Revised. Los Angeles, CA: Western Psychological Services; 2003. [Google Scholar]
  • 21.Mullen EM. Mullen Scales of Early Learning. AGS. Circle Pines, MN: American Guidance; 1995. [Google Scholar]
  • 22.Elliott CD. Differential Abilities Scale (DAS) San Antonio, TX: Psychological Corporation; 1990. [Google Scholar]
  • 23.Munson J, Dawson G, Sterling L, et al. Evidence of latent classes of IQ in young children with autism spectrum disorder. Am J Ment Retard. 2008;113:439–452. doi: 10.1352/2008.113:439-452. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Lord C, Risi S, DiLavore PS, Shulman C, Thurm A, Pickles A. Autism from 2 to 9 years of age. Arch Gen Psychiatry. 2006;63:694–701. doi: 10.1001/archpsyc.63.6.694. [DOI] [PubMed] [Google Scholar]
  • 25.Sparrow SS, Balla DA, Cicchetti D. Vineland Adaptive Behavior Scales. Circle Pines, MN: American Guidance; 1984. [Google Scholar]
  • 26.Stone WL, Ousley OY, Hepburn SL, Hogan KL, Brown CS. Patterns of adaptive behavior in very young children with autism. Am J Ment Retard. 1999;104:187–199. doi: 10.1352/0895-8017(1999)104<0187:POABIV>2.0.CO;2. [DOI] [PubMed] [Google Scholar]
  • 27.Autism and Developmental Disabilities Monitoring Network Surveillance Year 2006 Principal Investigators; Centers for Disease Control and Prevention (CDC) Prevalence of autism spectrum disorders—autism and developmental disabilities monitoring network, United States, 2006. MMWR Surveill Summ. 2009;58:1–20. [PubMed] [Google Scholar]
  • 28.Kogan MD, Blumberg SJ, Schieve LA, et al. Prevalence of parent-reported diagnosis of autism spectrum disorder among children in the US, 2007. Pediatrics. 2009;124:1395–1403. doi: 10.1542/peds.2009-1522. [DOI] [PubMed] [Google Scholar]
  • 29.Cohen D, Pichard N, Tordjman S, et al. Specific genetic disorders and autism: clinical contribution towards their identification. J Autism Dev Disord. 2005;35:103–116. doi: 10.1007/s10803-004-1038-2. [DOI] [PubMed] [Google Scholar]
  • 30.Badano JL, Teslovich TM, Katsanis N. The centrosome in human genetic disease. Nat Rev Genet. 2005;6:194–205. doi: 10.1038/nrg1557. [DOI] [PubMed] [Google Scholar]
  • 31.Bond J, Woods CG. Cytoskeletal genes regulating brain size. Curr Opin Cell Biol. 2006;18:95–101. doi: 10.1016/j.ceb.2005.11.004. [DOI] [PubMed] [Google Scholar]
  • 32.Courchesne E, Pierce K, Schumann CM, et al. Mapping early brain development in autism. Neuron. 2007;56:399–413. doi: 10.1016/j.neuron.2007.10.016. [DOI] [PubMed] [Google Scholar]
  • 33.Molloy CA, Keddache M, Martin LJ. Evidence for linkage on 21q and 7q in a subset of autism characterized by developmental regression. Mol Psychiatry. 2005;10:741–746. doi: 10.1038/sj.mp.4001691. [DOI] [PubMed] [Google Scholar]
  • 34.Parr JR, Lamb JA, Bailey AJ, Monaco AP. Response to paper by Molloy et al.: linkage on 21q and 7q in autism subset with regression. Mol Psychiatry. 2006;11:617–619. doi: 10.1038/sj.mp.4001833. author reply: 619. [DOI] [PubMed] [Google Scholar]
  • 35.Capone GT. Down syndrome: advances in molecular biology and the neurosciences. J Dev Behav Pediatr. 2001;22:40–59. doi: 10.1097/00004703-200102000-00007. [DOI] [PubMed] [Google Scholar]
  • 36.De Bildt A, Sytema S, Ketelaars C, et al. Interrelationship between Autism Diagnostic Observation Schedule-Generic (ADOS-G), Autism Diagnostic interview-Revised (ADI-R), and the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) classification in children and adolescents with mental retardation. J Autism Dev Disord. 2004;34:129–137. doi: 10.1023/b:jadd.0000022604.22374.5f. [DOI] [PubMed] [Google Scholar]
  • 37.Nordin V, Gillberg C. Autism spectrum disorders in children with physical or mental disability or both. I: clinical and epidemiological aspects. Dev Med Child Neurol. 1996;38:297–313. doi: 10.1111/j.1469-8749.1996.tb12096.x. [DOI] [PubMed] [Google Scholar]
  • 38.Bryson SE, Bradley EA, Thompson A, Wainwright A. Prevalence of autism among adolescents with intellectual disabilities. Can J Psychiatry. 2008;53:449–459. doi: 10.1177/070674370805300710. [DOI] [PubMed] [Google Scholar]
  • 39.Molloy CA, Murray DS, Kinsman A, et al. Differences in the clinical presentation of Trisomy 21 with and without autism. J Intellect Disabil Res. 2009;53(Part 2):143–151. doi: 10.1111/j.1365-2788.2008.01138.x. [DOI] [PubMed] [Google Scholar]
  • 40.Rowe J, Lavender A, Turk V. Cognitive executive function in Down's syndrome. Br J Clin Psychol. 2006;45:5–17. doi: 10.1348/014466505X29594. [DOI] [PubMed] [Google Scholar]
  • 41.Newschaffer CJ, Croen LA, Daniels J, et al. The epidemiology of autism spectrum disorders. Annu Rev Public Health. 2007;28:235–258. doi: 10.1146/annurev.publhealth.28.021406.144007. [DOI] [PubMed] [Google Scholar]

RESOURCES