Assessment of Speech Sound Production by Story-retelling in Persian Speaking Children: Introducing a New Instrument

Abstract

Background:

Speech and language pathologists should include connected speech assessment as part of their evaluation for children with speech sound disorders. The purpose of the present study was to design and validate an instrument for assessment of articulation by story-retelling for Persian children.

Methods:

261 typically developing children, aged 4-5 years old in Iran, Tehran, in 2016-2017, were recruited in the current study. First, two stories were designed. Next, the expert panel reviewed those two stories and selected one story for assessment. The strengths of the test for discriminating between the two age groups (48-54 months and 55-60 months) and between boys and girls were investigated for construct validity. Test-retest was performed for 15 children. Also, inter-rater reliability was evaluated via calculating the correlation between the two examiners’ scores. Statistical Package for Social Sciences (SPSS), version 24, was used for statistical analysis. The significance level was set at (P<0.05).

Results:

There was 80% or more agreement in experts’ response to questions regarding content validity. All of the reliability values were higher than 0.85. No significant difference was observed between boys and girls (P=0.77), but there was a significant difference between the two age groups (P<0.001). There was a significant correlation between this test and phonetic test of the Persian version of diagnostic articulation and phonology (r=0.62, P=0.002).

Conclusion:

The Persian story for the assessment of speech sound production is a reliable and valid instrument that can be used to evaluate the articulation of Persian children.

What’s Known

• There are some instruments for the assessment of connected speech in languages other than Persian. Due to differences between Persian and other languages, we cannot use those instruments for Persian speaking children.

What’s New

• In the current study, we designed an instrument for the assessment of connected speech for Persian children that has not been designed previously. We also investigated its psychometric properties.

Introduction

One of the potential qualities that distinguish humans from the rest of the social beings is the ability to communicate verbally with the members of the community. This verbal communication essentially requires a set of shared symbols as well as their actualization. Difficulty at the level of symbolization or actualization or both could result in a breakdown in verbal communication, the binding force of humans within a (linguistic) community. Any difficulties in symbolic (i.e., linguistic) skills could lead to incompetent communication.

The speech sounds, thus, bear an important role in verbal communication. Speech Sound Disorders (SSD) are the most common type of communication disorders in children.¹ SDD is a cover term that includes many speech disorders and is defined as the difficulty in the production of one or more speech sounds. Often, there is no obvious etiology for SSD.² Different rates are reported for the prevalence of SSD in the literature.^3,4 It is estimated that the prevalence of SSD is 13.1% for children in Iran.⁵ This signifies the need for tools to assess SSD in Persian children. SSD constitutes the major caseload of speech and language pathologists working with children.⁶ For instance, Mullen and Schooling reported that up to 56% of the clients in speech and language pathology services in schools were children with SSD.⁷ The age of most of the clients with SSD is under 8; however, occasionally these disorders persist to adulthood as well.³

There are some studies that report children with SSD are at increased risk of developing literacy and academic problems in the future.^8,9 So, there is always a need for comprehensive assessment of this population to design the best intervention protocol for them. Treatment and assessment of SSD have always been the specified role of speech and language pathologists (SLP). So, several attempts have been made to design and validate the assessment tools for assessing this population all over the world.¹⁰

One part of the speech sound evaluation is the assessment of speech sound production in connected speech. For a comprehensive assessment of children with SSD, assessment of speech sound production in connected speech should be performed since assessment of speech sound production in single word level does not give essentially sufficient information about speech sound capabilities of children with SSD because the ultimate aim in the treatment of children with SSD is the correct production of speech sound errors in connected speech.³

Klein and Liu-Shea’s study showed that the type and number of speech sound errors in connected speech sample were different from when these variables were checked in a single word assessment.¹¹

There are different procedures for elicitation of connected speech including spontaneous conversation, reading orally a passage, and story retelling task.³ The best and the most preferred method for the assessment of connected speech is obtaining spontaneous conversation. In this procedure, SLP engages the client in a spontaneous speech on topics like his or her family, TV programs, and so on. But, occasionally, obtaining a representative speech sample that includes all speech sounds is challenging and some children may use selective avoidance for some of speech sounds, meaning that the child misses words containing error sound. In cases we could not obtain this sample, other procedures can be used for the assessment of connected speech.

There is one subtest for the assessment of speech sound production in some familiar and popular tests like Goldman-Fristoe Test of Articulation (GFTA), second and third edition. Developers of GFTA designed two stories: one for children aged 4/0 to 6/11 years old with its pictures in cartoon style and another story for the age group of 7:0 to 21:11 years old with realistic style pictures.¹² In the Secord Contextual Articulation Tests, there are also separate stories for children and adults for the assessment of speech sound production,¹³ but there is only one story in phonological screening tool for Cantonese-speaking children.¹⁴ In this procedure, the child listens to each sentence the examiner utters and looks at the accompanying pictures; then the examiner asks him/her to retell that sentence.12 This delayed imitation is considered as a connected speech sample. There are a few assessment tools of speech sound production in single word level in Persian;^15,16 however, no instrument exists for the assessment of speech sound production in connected speech in Persian. So, we decided to design and validate an assessment tool that could measure speech sound production through story retelling and incorporated all Persian consonants and vowels.

Materials and Methods

The present study was a cross-sectional, methodological, and descriptive study. The current research was performed in 2016-2017 in Tehran city.

Participants

Participants were 261 typically developing children, aged 48-60 months, who were randomly selected from the nursery schools in Tehran, Iran. Munro stated that there is a need for 3-10 persons for each item so that we can investigate construct validity.¹⁷ Considering the number of items in the story (44 items), 211 children were recruited for the study for the investigation of construct validity. The sample size was calculated by this formula:

$n=\frac{Z_{(1-\frac{\alpha }{2})}^{2}P(1-P)}{d^2}$

α=0.05, d=0.04, P=Prevalence of speech sound disorders=13%, and n=261

Cluster method was used for sampling. Tehran city was divided into north, south, and center, and then the nursery schools were randomly selected. They were divided into two groups: group 1 (48-54 months) and group 2 (55-60 months). Inclusion criteria were Monolingual Persian speaking children aged 48-60 months. The exclusion criteria were visual and hearing impairment, delayed speech and language development, or facial disorders such as cleft lip and palate, autistic spectrum disorders, cerebral palsy, and mental retardation. The diagnosis was based on the examination and observations of an experienced speech and language pathologist, reports of parents and teachers in nursery schools, children’s medical records, and the available information obtained from a form that parents completed about different stages of communication, speech and language development of the child.

Item Generation

Stories were designed based on the existing resources through an extensive review. The major resources included were Persian phonetics,¹⁸ core vocabulary in Persian children,¹⁹ Persian translated McArthur-Bates Communicative Development Inventory speech and language development in Persian children,²⁰ development of phonetic inventory in Persian children,²¹ development of the consonant clusters in Persian children,²² phonetic information test,¹⁵ expression and perception of nouns in Persian children,²³ expressive lexicon in Persian children,²⁴ Persian picture naming test,²⁵ phonetic subtest of Persian version of diagnostic evaluation of articulation and phonology,¹⁶ and a modeling of Goldman-Fristoe test of articulation-third edition.¹² Firstly, the two stories were written. The first story was named ‘Birthday /tᴂvᴂllod/’ and the second ‘Painting /nᴂGaʃi/’. The attributes of these stories were as follows:

Both of them incorporated all Persian consonants and vowels and their subject matters and vocabularies were familiar to children. The mean length of sentences in two stories was between 3 to 4 words. They were relevant to the popular and familiar events for children (Birthday and Painting in class); stories that all children have experienced. The numbers of target words in the first and the second stories for the assessment of speech sound production in three positions (initial, medial, and final) of words were 47 and 44, respectively. The sentences were not lengthy, so a child could repeat them easily after the examiner. The written stories were sent to a professional graphic artist to prepare colorful and cartoon style pictures in size A4 for each story. The pictures were designed to assist children in retelling the story. The first story included 11 accompanying pictures and the second one included 10 pictures.

Content Validity

We asked the panel, comprising of three speech and language pathologists and two linguists who all had studied and worked on speech-sound acquisition and SSD, to review the two stories and answer the following questions using yes/no for questions 1-5 and select one story for question no. 6.

• 1. Do the stories include all the Persian speech sounds?

• 2. Are the stories adequately depicted?

• 3. Are the target words in the stories appropriate for the measurement of speech-sound production?

• 4. Are the topics of stories familiar to and interesting for 4-5 year-old children?

• 5. Are the pictures vivid and clear for children?

• 6. Which story is preferable for assessment of speech sound production for this age group?

The number of “yes” responses were considered as the percentage of agreement between experts. Experts had more agreement on the second story (painting), so this story was selected (Appendix 1).

Face Validity

To determine the face validity, 10 normally developing children, aged 4-5 years, were assessed using this tool to see whether children could easily imitate sentences and if the pictures were appropriate for the age group or not.

Construct Validity

One type of construct validity called known-groups validity was investigated for assessing of construct validity in the present study. This type of construct validity is related to the ability of the test to discriminate between the two groups or more considering the studied construct. The construct validity of the test (or known-groups validity) was evaluated by giving the test to the two age groups (48-54 months and 55-60 months) and both gender groups (boys and girls).²⁶ As for the evidence of construct validity, convergent validity, as the subtype of construct validity, was calculated by the calculation of the correlation between total scores of 40 children in the Persian story for speech-sound production and total scores of the phonetic test of the Persian version of diagnostic evaluation of articulation and phonology.²⁷

Reliability

For the test-retest reliability, the correlation between two administrations of the test on 15 children, randomly selected from the sample, with a two-week interval, was evaluated. For inter-rater reliability, two speech and language pathologists holding Ph.D. and MSc of Speech and Language Pathology, who were instructed about scoring and transcription procedure, transcribed and scored the recorded audio tracks. Internal consistency was calculated by the correlation between items of the test by using Kuder-Richardson 21.

Measures and Analyses

Statistical Package for the Social Sciences, version 24.0 (SPSS, Inc., Chicago, IL), was used for statistical analysis. We used the qualitative approach for content validity. Spearman correlation coefficient was used for convergent validity (because the distribution of the data was not normal, so the nonparametric test was used); Mann-Whitney test (because the distribution of the data was not normal) was run for known-groups validity; and Intra-class correlation coefficient was implemented to calculate inter-rater agreement and test-retest reliability. Kuder-Richardson test 21 was used for the internal consistency.

Scoring and Administration of the Test

The speech and language pathologist who had worked with children examined them. Assessment of each child was performed individually in a quiet room in the nursery school, which took about 15-20 minutes. The child sat on a chair in front of a table with its height appropriate for him/her to see the pictures easily. At first, the examiner explained to the child that she was going to tell him/her a story. He/she was asked to look at the accompanying picture and listen to what the examiner was saying simultaneously. After each sentence, the child retold what the examiner had told him/her. The examiner audio recorded the child’s speech sample and immediately transcribed it. A voice recorder (SONY-ICD-UX560F) and a laptop (ASUS-K45VD) were used to record speech samples for the transcription. If the child could produce the intended speech sounds for one word of the sentence correctly, score 0 was given to that sound in the record form and if the child had an error for that speech sound, score 1 was given to that sound in the record form. The maximum score given was 0, which was when the child could produce all the sounds correctly.

Ethical Considerations

The present study was approved by the Ethics Committee of Iran University of Medical Sciences (Code no. 1395.9221363201). We requested all the parents to complete the written informed consent form for participation of their children in the study. Children who did not cooperate with the examiner were eliminated from the study.

Results

Content Validity

The results of experts’ responses to questions related to the content validity of stories are given in Table 1. As shown in Table 1, the agreement percentage for each question was 80% or more. Considering “No’’ responses, some modifications were made to the pictures.

Table 1.

Experts’ responses for content validity of the story

Experts	Question 1		Question 2		Question 3		Question 4		Question 5		Question 6
	Story 1	Story 2	Story 1	Story 2	Story 1	Story 2	Story 1	Story 2	Story 1	Story 2
Expert 1	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	“Painting”
Expert 2	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	“Painting”
Expert 3	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	No	“Painting’’
Expert 4	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	‘’Birthday”
Expert 5	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	“Painting”

Face Validity

Children could easily imitate all sentences after the examiner and it seemed that the pictures were appropriate for them. Therefore, face validity of this assessment tool was regarded as good.

Descriptive Statistics

Participants’ descriptive statistics (n=211) in the two age groups, in terms of sex, is provided in Table 2. As shown in Table 2, there are 103 participants in group1 (48-54 months) and 108 in group 2 (55-60 months), among whom 100 participants were girls and 111 were boys.

Table 2.

Descriptive statistics of participants for the construct validity in terms of age and gender (n=211)

Age group	Girl	Boy	N (Total)
48-54 months	48	55	103
55-60 months	52	56	108
Total	100	111	211

Construct Validity

As given in Table 3, there was no significant difference in the mean values between girls and boys in mean values of speech-sound production in story retelling. However, according to Table 4, there was a significant difference between the mean values of the two age groups in speech-sound production of story retelling and there was a decrease in mean values of children in the older group. So, the test could discriminate between the two age groups (known-groups validity).

Table 3.

Mann-Whitney test to compare mean scores of the Persian test for assessment of the speech sound production in connected speech regarding gender (maximum score: 0)

Gender	Mean±SD	Z	P
Boy	1.70±1.78	-1.77	P=0.77
Girl	1.27±1.42

P<0.05

Table 4.

Man Whitney test to compare mean scores of Persian test for assessment of the speech sound production in connected speech regarding age (maximum score: 0)

Age group	Mean±SD	P
48-54 months	2.27±0.16	<0.001
55-60 months	0.75±0.11

There was a significant correlation between mean scores of 40 participants in this test and phonetic sub-test of the Persian version of the diagnostic evaluation of articulation and phonology for evidence of convergent validity (r=0.62, P=0.002).

Reliability Issues

We found a satisfactory correlation between test items. A good correlation was found between test items, which was shown by Kuder-Richardson test 21 (r=0.88, P<0.001).

Test-retest reliability was calculated and there was a significant correlation between participants’ mean scores in the two assessments (r=0.939, P<0.001).

Also, inter-rater reliability evaluation showed that there was a significant correlation between the two independent examiners’ scores (r=0.876, P<0.001).

Discussion

Introducing a new instrument for the assessment of speech sound production in connected speech for Persian children was an innovative aspect of the present study.

The content validity of this assessment tool was verified by the high agreement percentage between experts, which was higher than 0.8 for all questions. We designed two stories at first and then one was selected based on experts’ responses. The result was in line with those of the previous studies.^12,13,28 The stories were designed considering children’s age group. Painting and birthday are two events that are both objective and familiar to children. We tried to design the stories that included all Persian consonants and vowels and incorporated words that were appropriate for this age group. These criteria were in agreement with Goldman and Fristoe who designed a story that depicted a familiar event to children.¹²

The construct validity of this test was shown by the difference in the mean scores of children in the two age groups and boys and girls. The mean score for the older children was less than that of the younger ones, meaning that the ability of speech sound production in these age groups improved with age. This finding was in accordance with the reports by Goldman and Fristoe.¹² and many other studies on phonological development showing that as children grow up, their phonological skills expand.^29-31 Age is one of the major factors in speech sound production. It seems that maturation of oral motor function with age helps children produce speech sounds more accurately compared with younger children.³² Girls had better performance in this instrument regarding speech sound production, but there was no significant difference between the mean scores of boys and girls. This finding is in line with many international studies^31,33-36 and different from some others.^30,37 Up to now, there has been no consensus on the effect of sex on speech sound production. Hyde and Linn published a meta-analysis, including 170 studies, on the effect of sex on verbal abilities. They showed that sex constituted a relatively small variance (10-15%) in children’s speech sound production.³⁸ A high value was obtained for reliability (higher than 0.85). These findings are also in agreement with those reported in Persian^15,16 and other studies.^30,31 Also, a high reliability value was found for this instrument in inter-rater reliability test; and good value for the internal consistency shows that all items measured a single performance.

There were also some limitations in the present study that should be mentioned before generalizing the findings. First, the study was performed only on 4-5 year-old children in two age groups and other age groups were not studied. The current study was part of a more comprehensive study, a Ph.D. dissertation that participants were 4-5 years old. Second, we did not include children with SSD. Comparing the performance of children with SSD and that of typically developing children by this instrument could increase the clinical importance of this tool in identifying children with speech impairment. Third, standard scores were not reported in the current study. Further studies should be conducted to investigate the administration of this tool on children with SSD and provide normative data on a large sample of children in several age groups. There are four phases for studies on item development. Phase 1 is called item generation by which items of instrument are generated. In Phase 2, or validity and reliability, the psychometric properties of the measurement are investigated. Phase 3 is normalization in which standard scores of the tools are reported; and in phase 4, or application, the tool is used for different groups of disordered persons.¹⁷ The first two phases are essential for each instrument development, and they were performed in the current study. We will conduct the other phases in further studies.

Conclusion

It seems that the Persian story of speech-sound production in connected speech is a valid and reliable assessment tool for 4-5 year-old children. It can be used to evaluate the speech sound production abilities in Persian-speaking children.