Abstract
Objective
To evaluate two methods for identifying speech and language problems in preschool children.
Design
Prospective population based study.
Setting
Inner London.
Participants and methods
37 health visitors were randomly assigned to use a structured screening test (18) or a parent led method (19). Of 623 eligible children aged 30-36 months, the parents of 582 agreed to participate (353 using the structured test and 229 the parent led method).
Main outcome measures
Children were assessed by a speech and language therapist blinded to the test result, using the Reynell developmental language scales. Children were classified as having “severe language problems” if the Reynell score was below the third centile for receptive language and as “needing therapy” if the Reynell score was below the seventh centile for receptive or expressive language and clinical opinion.
Results
Reference assessments and usable scores were obtained for 458 (97%) of the 474 children screened. 98 (21%) children had severe language problems and 131 (29%) needed therapy. The sensitivity and specificity for the structured screening test were 66% (95% confidence interval 53% to 76%) and 89% (85% to 93%) respectively for severe language problems and 54% (43% to 65%) and 90% (85% to 93%) for those needing therapy. The sensitivity and specificity for referral by the parent led method were 56% (40% to 71%) and 85% (78% to 90%) for severe language problems and 58% (44% to 71%) and 90% (83% to 94%) for those needing speech and language therapy.
Conclusions
Both approaches failed to detect a substantial proportion of children with severe language problems and led to over-referral for diagnostic assessments. Screening is likely to be an ineffective approach to the management of speech and language problems in preschool children in this population.
What is already known on this topic
Moderate to severe language difficulties in young children are predictive of long term problems affecting learning, school achievement, and behaviour
Formal screening tests are widely used, but relying on parents' observations and health professionals' clinical judgment may be more effective in identifying children needing therapy
What this study adds
A commonly used screening test and an approach based on parents' observations and health visitors' judgment fail to identify a substantial proportion of children with serious language problems and lead to the over-referral of children without serious difficulties
Introduction
Moderate to severe language difficulties in young children, particularly those affecting language comprehension, are predictive of long term problems affecting learning, school achievement, and behaviour.1–5 Young children presenting with language difficulties include those with specific impairment of language, autistic spectrum disorder, and general learning difficulties, as well as those whose difficulties are familial or associated with environmental factors.6,7 For an individual child, the prognosis will depend on the cause of the problem and its responsiveness to intervention.8,9
The rate at which children acquire language varies greatly, and the definitions of “language delay” and “specific language impairment” lack clarity. The development of communication skills is closely linked with other areas of development—such as cognitive, social, and behavioural skills—which in turn influence language development. Estimates of the prevalence of speech and language impairment depend on the cut-off point chosen in a range across several component skills. Estimates of the prevalence of language difficulty in preschool children are between 1% and 19%.10
Most children with severe language difficulties are probably identified before they start school, but many may not be identified until they start formal education. Questions have been raised about the effectiveness of formal screening, although programmes exist in many developed countries and were common in the United Kingdom until recently.11–14 Evidence suggests that parents are good observers of their child's development and that parents' reports may be as good as screening in detecting developmental difficulties.15,16 As a result, many child health surveillance programmes in the United Kingdom now give advice about language development and rely on parents to identify children with problems in consultation with health visitors.12,17
The reliance on parents' observations and health visitors' clinical judgment is philosophically attractive, but the effectiveness of using this approach to identify children with serious speech and language difficulties is unclear. We compared the performance of a structured screening test with a method based on parental concerns and health professionals' clinical judgment against a reference assessment of speech and language in preschool children.
Methods
We invited all health visitors working in City and Hackney District, London, to participate in the study, and we assigned numbers to those who agreed. A statistician independent of the study used these numbers to randomly allocate health visitors in blocks of two (to ensure equal numbers) to either the structured screening or the parent led method. The health visitors received training in their allocated method. We asked the health visitors to invite parents of all children attending for their developmental check at age 2½ years to participate in the study whether or not the child had a previously diagnosed developmental problem (confined to children aged 30-36 months seen as part of the routine child health promotion programme between February 1998 and June 1999). We obtained consent from the local research ethics committee.
We estimated the sensitivity of the formal screening test at 90%, that of the parent led method at 70%, and the prevalence of speech and language problems at 10%. From the number of eligible children and known uptake rates, we expected that health visitors would recruit 800 children in one year (400 in each group). This number would have implied a standard error of about 3.5% in the estimated sensitivity in the formal screening group and 5% in the parent led method. Because recruitment was lower than expected, we continued for a further five months, while funding was available.
The structured screening test contained 20 questions covering both expressive and receptive language skills and had been previously validated on the local population.18 A point was awarded for each question and a score of below 10 was regarded as a fail.
In the parent led method, we asked health visitors to discuss with parents whether they had any concerns about their children's language development. From this discussion and using clinical judgment based on history and observation, health visitors decided whether to refer the child, review the child at a later date, or classify the child as having no difficulties.
The assessment was carried out as part of the routine check, and parents were then asked to see one of two research speech and language therapists, either immediately or at an appointment within two weeks. The speech and language therapists carried out a diagnostic assessment based on the Reynell developmental language scales.19 This test is a comprehensive assessment of language skills and provides centiles standardised by age for expressive and receptive language. Each therapist saw children screened by both methods and were blinded to the outcome of the health visitor's assessment. Parents were asked not to tell the speech and language therapist the results of the screening assessment.
We classified children scoring less than the third centile on the Reynell receptive language scale as having severe language problems and children scoring below the seventh centile in either expressive or receptive language or who, in the opinion of the therapist, would benefit from referral as requiring speech and language therapy.
We obtained the number of children eligible for their developmental check at age 2½ years and the proportion to be enrolled into each arm of the study from the Regional Child Health System.
Analysis
We entered data into EpiInfo.20 We calculated sensitivity, specificity, and likelihood ratios for each method for the two diagnostic methods. For sensitivity and specificity we calculated 95% confidence intervals using the exact binomial method and for likelihood ratios we used a method recommended by Simel et al.21,22 This study was originally designed as a cluster randomised controlled trial to compare the performance of the two screening methods. As neither performed adequately compared with the reference, it was decided that no statistical comparison should be made between the two methods.
Results
Of 47 health visitors, 37 agreed to participate. In all, 18 health visitors were allocated to the structured screening group and 19 to the parent led group. During the study period, health visitors did developmental checks on 1598 children aged 2½ years, 40% of the total population. Of these, 308 (19%) were checked by a professional who was not taking part in the study. Similar proportions of children attending for checks with participating health visitors were enrolled into the study in each group. In the structured screening group, 376 children were invited to participate, of whom 353 (94%) agreed; 291 (77%) attended for screening and diagnostic assessment and usable results were obtained for 282 (75%). In the parent led group, 247 children were invited to participate, of whom 229 (93%) agreed, 183 (74%) attended for screening and diagnostic assessment, and usable results obtained for 176 (77%) (figure).
The number of children recruited varied among health visitors (range 1-37; mean 16.2). The characteristics of the health visitors in each group and the children who agreed to participate and of those included in the analyses are given in table 1.
Table 1.
Characteristics of participants
| Children initially recruited (n=582)
|
Children completing study
|
||
|---|---|---|---|
| Structured test (n=282)
|
Parent led method (n=176)
|
||
| Mean (range) age (months) | 32 (30-36) | 32 (30-36) | 32 (30-35) |
| No (%) of boys | 326 (56) | 158 (56) | 89 (51) |
| No (%) whose first language is English | 474 (81) | 236 (84) | 151 (86) |
Of the 458 children who completed the reference assessment, 98 (21%) scored below the third centile and 131 (29%) were judged as “needing therapy.” Performance in screening tests and reference assessments with likelihood ratios and 95% confidence intervals is given in tables 2 to 5.
Table 2.
Results for structured screening method and comprehension score on Reynell developmental language scales. Values are numbers of children, unless otherwise stated
| Score on screening test*
|
Reynell score
|
Total
|
Likelihood ratio (95% CI)
|
|
|---|---|---|---|---|
| Below 3rd centile
|
Above 3rd centile
|
|||
| <10 | 42 | 23 | 65 | 6.22 (4.07 to 9.52) |
| ⩾10 | 22 | 195 | 217 | 0.38 (0.27 to 0.54) |
| Total | 64 | 218 | 282 | |
A score of 10 or less was regarded as a fail.
Table 5.
Results for parent led screening method compared with health visitor's decision whether child needs therapy*
| Result of screening consultation
|
Needs therapy
|
Does not need therapy
|
Total
|
Likelihood ratio (95% CI)
|
|---|---|---|---|---|
| Refer | 28 | 13 | 41 | 5.74 (3.25 to 10.14) |
| Review | 10 | 20 | 30 | 1.33 (0.67 to 2.64) |
| No concerns | 10 | 95 | 105 | 0.28 (0.16 to 0.49) |
| Total | 48 | 128 | 176 |
Based on a comprehension score on the Reynell scale below the seventh centile or a speech therapist's decision that a child would benefit from therapy.
The sensitivity and specificity for the structured screening test were 66% (95% confidence interval 53% to 76%) and 89% (85% to 93%) for children with severe language problems and 54% (43% to 65%) and 90% (85% to 93%) for children needing therapy. When children who scored more than 10/20 whose parents or health visitors were concerned were included in the fail category, the sensitivity increased slightly but the specificity decreased.
The sensitivity and specificity for referral by the parent led method were 56% (40% to 71%) and 85% (78% to 90%) for severe language problems and 58% (44% to 71%) and 90% (83% to 94%) for children needing therapy. We also combined the referral and review categories to reflect those children causing some concern. The resulting sensitivity and specificity were 77% (60% to 88%) and 68% (60% to 75%). for children with severe language problems and 79% (66% to 88%) and 74% (66% to 81%) for children needing therapy.
Analysis of the data including only the 387 (85%) children whose first language was English made virtually no difference to the results. Of these children, 72 (19%) scored below the third centile on the Reynell receptive language scale.
Discussion
In the United Kingdom, it has been argued that formal screening for speech and language difficulties in preschool children should be replaced by an approach in which health professionals respond to parental concerns.12 The effectiveness of this approach has not been assessed previously. We asked health visitors using the parent led method to elicit parental concerns about their children's speech and language and to interpret these concerns in the light of their clinical knowledge. Health visitors allocated to this method attended refresher workshops to ensure that they felt comfortable with what would be considered normal language development in children of this age group. The results are disappointing; at least within this deprived inner city population, the approach would fail to identify a substantial proportion of children with serious language problems, while also leading to over-referral of children without difficulties.
Our results do not, however, imply that formal approaches to screening are best. The structured screening test performed as poorly as the parent led method. A previous assessment of this test in the same area had reported a higher sensitivity, but in that study only some children whose screen showed no language difficulties could be assessed.18
The sensitivity and specificity needed by a potential screening test depends on context and the balance between the consequences of correct and incorrect classification and the underlying prevalence of the condition. The performance of the approaches we tested is clearly less than satisfactory. We therefore decided that statistical comparison of performance between the tests would be inappropriate.
Applicability of results
The extent to which these results are applicable to other populations is unclear. The application of estimates of test performance from one population to another depends on the relation between the range of abnormality in the two groups. This is likely to differ between more and less deprived areas. Hackney is a deprived area where many people speak languages other than English. Restriction of the analyses to those children whose first language was English, however, made no difference to the results. The results apply most directly to children in other deprived areas.
A further difficulty is that the low uptake of the developmental check at age 2½ years and attrition mean that we cannot assume our sample is representative of children in the district. Children whose parents were anxious about their language development may have been disproportionately likely to participate, and, similarly, health visitors may have preferentially enrolled children whom they were concerned about, which would accord with the high prevalence of serious language problems. This would have biased the study towards overestimating the performance of the screening tests.
Treating or preventing speech and language difficulties
The rationale for trying to identify children with speech and language difficulties is the belief that early intervention by speech and language therapists can improve outcome. A systematic review published in 1998 found that although there are many studies showing positive effects of interventions for speech and language therapy, the evidence was relatively limited.10 A recent randomised controlled trial suggested that therapy had little effect, although the trial was criticised because of the small amount of therapy provided to the intervention group.23,24 This level of therapy is, however, likely to reflect the amount of therapy available in many NHS facilities in the United Kingdom. The effectiveness of therapy is likely to depend on the specific problem and on the presence of comorbidities and other contributory factors.
Early speech and language difficulties are strongly associated with later adverse outcomes.1–5 Our results suggest that using formal screening or a parent led approach to identify children with speech and language difficulties is unlikely to be effective. Little consensus about alternative approaches, however, has been reached. The factors determining speech and language development are not understood, and children with problems include those with multiple developmental problems; those with specific possibly genetically determined, communication disorders; those where the problems are associated with poor environmental conditions or neglect; and many for whom the cause is unclear.6,7 Retaining developmental screening may be useful in areas of social disadvantage because parents may be less likely to successfully obtain support services for children with impairments. A more effective instrument, however, than the screening test evaluated here would be needed.25
Population based approaches to the promotion of language development are intuitively attractive, particularly in areas with a high prevalence of children with serious problems. In line with this approach, initiatives such as Sure Start in the United Kingdom have resulted in a wide range of innovative community based approaches designed to target speech and language development.26 Unfortunately the evidence base for these proposed interventions is relatively weak. Better evaluation of such interventions should be a matter of priority.
Figure.
Flow of children through the study
Table 3.
Number of children needing treatment from the structured screening method*
| Score on screening test†
|
Needs therapy
|
Does not need treatment
|
Total
|
Likelihood ratio (95% CI)
|
|---|---|---|---|---|
| <10 | 45 | 20 | 65 | 5.39 (3.40 to 8.55) |
| ⩾10 | 38 | 179 | 217 | 0.51 (0.40 to 0.65) |
| Total | 83 | 199 | 282 |
Comprehension score on Reynell scale below the seventh centile or if the speech therapist thought therapy was necessary.
A score of 10 or less was regarded as a fail.
Table 4.
Results for parent led screening method and comprehension score on Reynell developmental language scales
| Result of screening consultation
|
Reynell score
|
Total
|
Likelihood ratio (95% CI)
|
|
|---|---|---|---|---|
| Below 3rd centile
|
At least 3rd centile
|
|||
| Refer | 19 | 22 | 41 | 3.61 (2.22 to 5.87) |
| Review | 7 | 23 | 30 | 1.27 (0.60 to 2.71) |
| No concerns | 8 | 97 | 105 | 0.34 (0.19 to 0.64) |
| Total | 34 | 142 | 176 | |
Acknowledgments
We thank the health visitors, parents, and children who participated; Julian Higgins, who did the randomisation; and Bernard Camilleri, who performed diagnostic assessments on some of the children.
Footnotes
Funding: The Inner City Research and Development Group of the former North Thames regional office of the NHS Executive.
Competing interests: None declared.
References
- 1.Stothard SE, Snowling MJ, Bishop DVM, Chipchase BB, Kaplan CA. Language impaired pre-schoolers: a follow-up into adolescence. J Speech Lang Hear Res. 1998;41:407–418. doi: 10.1044/jslhr.4102.407. [DOI] [PubMed] [Google Scholar]
- 2.Johnson CJ, Beitchman JH, Young A, Escobar M, Atkinson L, Wilson B, et al. Fourteen-year follow-up of children with and without speech/language impairments: speech/language stability and outcomes. J Speech Lang Hear Res. 1999;42:744–761. doi: 10.1044/jslhr.4203.744. [DOI] [PubMed] [Google Scholar]
- 3.Conti-Ramsden G, Botting N, Simklin Z, Knox E. Follow-up of children attending infant language units: outcomes at 11 years of age. Int J Lang Commun Disord. 2001;36:207–221. [PubMed] [Google Scholar]
- 4.Aram DM, Nation JE. Preschool language disorders and subsequent academic difficulties. J Commun Disord. 1980;13:150–159. doi: 10.1016/0021-9924(80)90033-7. [DOI] [PubMed] [Google Scholar]
- 5.Silva PA, Williams SM, Mcgee R. A longitudinal study of children with developmental language delay at age three: later intelligence, reading and behaviour problems. Dev Med Child Neurol. 1987;29:630–640. doi: 10.1111/j.1469-8749.1987.tb08505.x. [DOI] [PubMed] [Google Scholar]
- 6.Cohen NJ, Lipsett L. Recognised and unrecognised language impairment in psychologically disturbed children: child symptomology, maternal depression and family dysfunction: preliminary report. Can J Behav Sci. 1991;23:376–389. [Google Scholar]
- 7.Law J, Conway J. Child abuse and neglect: the effect on communication development, a review of the literature. London: Afasic; 1992. [Google Scholar]
- 8.Paul R, Cohen D, Caparulo BK. A longitudinal study of patients with severe developmental disorders of language learning. J Am Acad Child Psychiatry. 1983;22:525–534. doi: 10.1097/00004583-198311000-00002. [DOI] [PubMed] [Google Scholar]
- 9.McRae KM, Vickar E. Simple developmental speech delay: a follow-up study. Dev Med Child Neurol. 1991;33:868–874. doi: 10.1111/j.1469-8749.1991.tb14795.x. [DOI] [PubMed] [Google Scholar]
- 10.Law J, Boyle J, Harris F, Harkness A, Nye C. Screening for speech and language delay: a systematic review of the literature. Health Technol Assess. 1998;2:1–184. [PubMed] [Google Scholar]
- 11.Hall DMB. Health for all children. 2nd ed. Oxford: Oxford University Press; 1991. [Google Scholar]
- 12.Hall DMB. Health for all children. 3rd ed. Oxford: Oxford University Press; 1996. [Google Scholar]
- 13.American Academy of Pediatrics. Screening infants and young children for developmental disabilities. Pediatrics. 1994;93:863–865. [PubMed] [Google Scholar]
- 14.De Winter M, Balledux M, de Mare J, Burgmeijer R. Screening in child health care. Report of the Dutch working party on child health care. Oxford: Radcliffe Medical Press; 1995. [Google Scholar]
- 15.Glascoe FP. Do parents discuss concerns about children's development with health care providers? Ambulatory Child Health. 1997;2:349–356. [Google Scholar]
- 16.Glascoe FP. Evidence-based approach to developmental and behavioural surveillance using parents' concerns. Child Care Health Dev. 2000;26:137–149. doi: 10.1046/j.1365-2214.2000.00173.x. [DOI] [PubMed] [Google Scholar]
- 17.Hall D, Elliman D. Health for all children. 4th ed. Oxford: Oxford University Press; 2002. www.health-for-all-children.co.uk/hfac4_draft.htm (accessed 13 Sep 2002). [Google Scholar]
- 18.Law J. Early language screening in City and Hackney: the concurrent validity of a measure designed for use with 2.6 year olds. Child Care Health Dev. 1994;20:295–308. doi: 10.1111/j.1365-2214.1994.tb00392.x. [DOI] [PubMed] [Google Scholar]
- 19.Edwards S, Fletcher P, Garman M, Hughes A, Letts C, Sinka I. The Reynell developmental language scales III. Windsor: NFER-Nelson; 1997. . (University of Reading edition.) [DOI] [PubMed] [Google Scholar]
- 20.Dean JA, Coulombier D, Smith DC, Brendel KA, Arner TG, Dean AG. EpiInfo v 6.03: a word processing, database and statistics programme for epidemiology on microcomputers. Atlanta: Centers for Disease Control and Prevention; 1995. [Google Scholar]
- 21.Bryant T. Confidence interval analysis. Southampton: University of Southampton; 2000. [Google Scholar]
- 22.Simel DL, Samsa GP, Matchar DB. Likelihood ratios with confidence: sample size estimation for diagnostic test studies. J Clin Epidemiol. 1991;44:323–338. doi: 10.1016/0895-4356(91)90128-v. [DOI] [PubMed] [Google Scholar]
- 23.Glogowska M, Roulstone S, Enderby P, Peters TJ. Randomised controlled trial of community based speech and language therapy in preschool children. BMJ. 2000;321:923. doi: 10.1136/bmj.321.7266.923. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Law J, Conti-Ramsden G. Treating children with speech and language impairments. BMJ. 2000;321:908–909. doi: 10.1136/bmj.321.7266.908. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Alberman E, Goldstein H. The “at risk” register: a statistical evaluation. Br J Prev Soc Med. 1970;24:129–135. doi: 10.1136/jech.24.3.129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Law J, Harris F. Promoting speech and language development: guidance for Sure Start programmes 2001. Nottingham: Department for Education and Skills. www.surestart.gov.uk/infoDocs/18114spe.&_lang.pdf (accessed 28 Oct 2002).