Feasibility of using a handheld electronic device for the collection of patient reported outcomes data from children

Abstract

The manner in which a communication disorder affects health-related quality of life (QOL) in children is not known. Unfortunately, collection of quality of life data via traditional paper measures is labor intensive and has several other limitations, which hinder the investigation of pediatric quality of life in children. Currently, there is not sufficient research regarding the use of electronic devices to collect pediatric patient reported outcomes in order to address such limitations. Thus, we used a cross-over design to compare responses to a pediatric health quality of life instrument (PedsQL 4.0) delivered using a handheld electronic device to those from a traditional paper form. Respondents were children with (n=9) and without (n=10) a speech or voice disorder. For paper versus the electronic format, we examined time to completion, number of incomplete or inaccurate question responses, intra-rater reliability, ease of use, and child and parent preference. There were no significant differences between children’s scores, time to complete the measure, or ratings related to ease of answering questions. The percentage of children who made answering errors or omissions with paper and pencil was significantly greater than the percentage of children who made such errors using the device. This preliminary study demonstrated that use of an electronic device to collect QOL or patient-reported outcomes (PRO) data from children is more efficient than and just as feasible, reliable, and acceptable as using paper forms. The development of hardware and software applications for the collection of QOL and/or PRO data in children with speech disorders is likely warranted.

1. Introduction

Patient reported outcome (PRO) measures are used in conjunction with more traditional measures of health and disease as additional and important clinical endpoints, and to evaluate the extent to which a treatment has affected the status of a patient’s health (Wiklund, 2004). Recent interest in the study of health-related quality of life (HR-QOL), a subset of measures falling under the umbrella of PROs, has led to the need for development of improved data collection methods and more study of HR-QOL in children (Kaplan, 2005). Likewise, while many HR-QOL instruments have recently been developed for use in children and adolescents, (Erickson, Montague, & Gerstle, 2010; Borton, Mauze, & Leiu, 2010; Garma, Kelly, Daharsh, & Vogel, in press; Gates, Otsuka, Sanders, & McGee-Brown, 2010; Ravens-Sieberer, Erhart, Willie, Wetzel, Nickel, & Bullinger, 2006; Bullinger & Ravens-Sieberer, 1995; Zur, Cotton, Kelchner, Baker, Weinrich, & Lee, 2007), more ways to easily use and score these instruments are necessary.

Although PRO data are viewed as important to the healthcare process, they are still underutilized (Drummond, Ghosh, Ferguson, Brackenridge, & Tiplady, 1995). This is particularly true with regard to children. Previous studies have indicated that pediatric HR-QOL must be studied further because children may perceive health quality differently than adults, and parent proxy may not be an adequate representation of children’s perceptions of health and disease (Gates, Otsuka, Sanders, & McGee-Brown, 2010; Varni & Limbers, 2009; Varni, Burwinkle, & Lane, 2005; Varni, Seid, & Kurtin, 2001; Ruscello, Lass, & Podbesek, 1988; Lass, Ruscello, Harkins, & Blankenship, 1993; Lass, Ruscello, Stout, & Hoffman, 1991; Moran, LaBarge, & Haynes, 1988; Connor, Cohen, Theis, Thibeault, Heatly, & Bless, 2008; Markham, van Laar, Gibbard, & Dean, 2009; Verduyckt, Remacie, Jamart, Benderitter, & Morsomme, 2011).

Voice and speech disorders, may negatively affect HR-QOL in children (Zur, Cotton, Kelchner, Baker, Weinrich, & Lee, 2007), as has also been indicated for other types of communication impairments based on parent proxy measures (Markham and Dean, 2006). For example, based on acoustic recordings alone, children with voice problems (dysphonia) or speech impairments (dysarthria) were rated more negatively on physical, social, and personality factors when compared with children without voice or speech impairments (Lass, Ruscello, Harkins, & Blankenship, 1993; Lass, Ruscello, Stout, & Hoffman, 1991; Ruscello, Lass, & Podbesek, 1988). Specifically, dysphonic or dysarthric children were rated more negatively on many bipolar comparisons of physical and personality characteristics, such as “dirty-clean,” “bad-good,” “cruel-kind,” “worthless-valuable,” “dishonest-honest,” “sick-healthy,” “sad-happy,” “wrong-right.” The previously mentioned studies indicate that children’s speech and voice disorders impact how they are perceived, which may affect how they are treated, and possibly result in compromised health related quality of life. This information supports a need for research specifically examining health related quality of life in children with voice and speech disorders. Some studies of children with dysphonia indicate the specific ways that children may be impacted by their disorder. For example, children with chronic dysphonia, ages 2-18 reported a negative impact on their lives across the domains of physical, social/functional, and emotional performance (Connor, Cohen, Theis, Thibeault, Heatly, & Bless, 2008). In a similar study, children with dysphonia, ages 5-13, were significantly more likely to express 17 out of 27 complaints related to their physical, emotional, and social functioning than children without dysphonia in this age range (Verduyckt, Remacie, Jamart, Benderitter, & Morsomme, 2011). While these studies illuminate that children are often affected negatively by their speech and voice disorders, the relative dearth of research directly examining quality of life in children with speech or voice disorders is problematic. With the improvement of PRO collection methods, pediatric quality of life may be highlighted with greater frequency in research, and, in turn, encourage the integration of PRO collection in speech clinics.

Current measures of pediatric quality of life have differentiated between healthy and ill children. Specifically, in a large-scale study of PROs in children, the Pediatric Quality of Life Inventory (PedsQL 4.0), a measure of the perception of general health quality for children ages 5 to 18, was administered to 2436 children (ages 8 to 18) and 4227 parents. The parent questionnaire required mothers and fathers to assess their child’s quality of life and thus served as a proxy measure. The PedsQL 4.0 has child self-report versions for ages 5-7, 8-12, 13-18. Parent proxy is available in versions for the latter age breakdowns as well as ages 2-4. These measures have accurately distinguished between healthy children and those with chronic health conditions (Varni, Burwinkle, & Seid, 2006).

New methods for electronic data capture (EDC) of PROs have exhibited overall favorability and efficiency compared to paper methods in studies using these methods with adults (Athale, Sturley, Skoczen, Kavanaugh, & Lenert, 2004; Wilson, Kitas, Carruthers, Reay, Skan, Harris, Treharne, Young, & Bacon, 2002). Based on this information, EDC is more likely to make the collection of PROs feasible in clinical practice and research involving adult populations. Still, it is not clear if PRO collection from children by EDC will exhibit the same positive benefits. Yet, paper administrations of PRO measures are time-consuming because they require data manipulation and scoring prior to interpretation. Thus, the underutilization of PROs may be partially related to the labor-intensive nature of scoring and distributing many paper PROs measures. Therefore, improvements in ease of inclusion of these measures may also improve utilization. EDC may also be beneficial because it allows for immediate scoring and generation of a summary page of PRO scores for inclusion in a spreadsheet of data for research purposes, or a patient’s medical record (Athale, Sturley, Skoczen, Kavanaugh, & Lenert, 2004; Wilson, Kitas, Carruthers, Reay, Skan, Harris, Treharne, Young, & Bacon, 2002).

In clinical care, EDC reduces the amount of time for information to flow from the patient to the practitioner (Wilson, Kitas, Carruthers, Reay, Skan, Harris, Treharne, Young, & Bacon, 2002). In research, EDC is likely to make large-scale collection of PROs more feasible and decrease hours of labor time. However, there is concern that PROs collected over the Internet for either research or clinical practice are more likely to represent only one sector of society. In a previously published article on this topic, it was found that adults who responded to a health related quality of life (HR-QOL) instrument via the Internet were younger, wealthier, and more educated than those who did not respond (Soetikno, Mrad, Pao, & Lenert, 1997). Accordingly, individuals who have the ability and opportunity to use a computer for questionnaire responses may be different from the overall population of patients (Treadwell, Soetikno, & Lenert, 1999) and, specifically, different from individuals who live in poverty or in remote, underserved areas of the United States.

The use of handheld devices, such as personal data assistants (PDAs) is an alternative to the use of computer and Internet methods for the collection of PRO data. PDA collection of PROs may better represent the overall population of interest because such devices may be mailed or delivered to families living in remote areas, individuals without computers in their home, or school settings. PDAs show promise in reducing response errors, improving response rates, and decreasing recall bias and missing responses, especially when responses across multiple days or time points are required (Drummond, Ghosh, Ferguson, Brackenridge, & Tiplady, 1995; Bushnell, Reilly, Galani, Martin, Ricci, Patrick, & McBurney, 2006; Stone, Shiffman, Schwartz, Broderick, & Hufford, 2002; Harding, Hamm, Ehsanullah, Heath, Sorrells, Haw, Dukes, Wolfe, Mangel, & Northcutt, 1997; Stinson, 2009; Broderick, 2008). Because of their portability, PDAs may be helpful to researchers and clinical practitioners who would like to collect PROs in vulnerable and underrepresented populations such as children in remote or impoverished areas (Hahn & Cella, 2003). However, PDAs may also be overly complex, and difficult for children to use when responding to PRO instruments. Therefore, dedicated computer software or devices that focus on use with children should be developed and tested.

Because electronic PROs collection in children both with and without communication disorders is not well studied, we explored whether it was feasible to use an electronic handheld device to collect PRO/QOL information from children. The aim of this preliminary study was to determine whether the efficiency, acceptability, and reliability of children’s responses to the Pediatric Quality of Life Inventory (PedsQL 4.0) implementation via an electronic interface and the traditional paper version were comparable when used by children ages 8-12 with and without speech and/or voice disorders. It was not our goal to compare HR-QOL in children with communication impairments to typically developing children. Rather, we wished to establish that collecting patient reported outcomes electronically in children with and without speech disorders was equivalent to their collection by paper and pencil forms.

2. Methods

2.1. Participants

Twenty children (10 typically developing and 10 with speech or voice disorders), ages 8-12, were recruited as participants for this comparison study through campus and local newspapers accessible to residents in Madison, Wisconsin. Children with voice and speech disorders were identified by parent report after they responded to the ad for the study. For inclusion, parents had to indicate that their child was currently receiving speech services in school or privately or had received those services within the past 6 months. Children also had to pass pure tone audiometry screening at 500, 1000, 2000, and 4000 Hz at 20 decibels, and the Clinical Evaluation of Language Fundamental-4 (CELF-4) (Semel, Wiig, & Secord, 2004) screening in order to be included in the study. Hearing and language screenings were administered by a licensed, certified speech-language pathologist during the first of the two sessions of the study. One child with a speech disorder did not pass pure tone audiometry screening decreasing the number of children in the study to 19. Statistical power for this sample size was calculated to be 80%.

Of the 19 children, 13 were male (68%) and 6 were female (32%). Four of the children (21%) were 8 years old, 6 (32%) were 9 years of age, 4 (21%) were 10 years of age, 4 (21%) were 11 years of age, and 1 child (5%) was 12 years of age. Ten of the nineteen children (53%) were typically developing, while 9 children (47%) had a speech or voice disorder.

Using a crossover comparison study design, children and parents visited the University of Wisconsin laboratory twice over a three-week period. The crossover component involved use of a traditional paper form on one visit and use of a custom developed handheld electronic device on the other visit. The study visits modeled a clinical encounter in which parents and children visited a hearing and speech clinic, rather than a remote use encounter where devices would be mailed to the home. We developed the device and software for this initial feasibility study rather than using a commercially available mobile device application. Doing so, allowed us to control all parameters of device operation and remove other functions not needed for questionnaire delivery.

On visit 1, participants were given the electronic implementation of the twenty-three item PedsQL 4.0 or the PedsQL 4.0 paper form followed by a one-week washout period. Then, on visit 2, participants were given the test modality not provided in their first visit. Thus, participants were divided randomly into two groups. In the first group, 10 children were given the paper version of the PedsQL 4.0 (Varni & Limbers, 2009; Varni, Burwinkle, & Seid, 2006; Varni, Seid, & Kurtin, 2001) for ages 8-12 and asked to respond to all twenty-three questions. For the second group of 9 children and parents, the same procedures described for Group 1 were used but in reverse order. Five typically developing children were in each group while 4 children with voice or speech impairment were in one group and 5 children with a speech or voice disorder were in the other. Randomization was based on a previously determined randomization schedule.

Measures of efficiency, reproducibility, and acceptability were collected during administration of the PedsQL 4.0. To reflect efficiency, the time to complete the paper and electronic versions of the PedsQL 4.0 was recorded. Additionally, the percentage of children who skipped or answered questions with errors was compared between implementation types. Thus, the percentage of children who produced question errors or omissions along with time to completion reflected efficiency. To examine reproducibility, intraclass correlation coefficients between scores on the PedsQL 4.0, obtained using the electronic prototype and the traditional paper form, were calculated.

For acceptability assessment, parents and children were asked to rank the: (1) ease of reading test items; (2) ease of responding; and (3) ease of changing a response for both administration methods. Children and parents were asked to mark a 100 mm visual analog scale (VAS), with zero indicating “easy” and 100 indicating “difficult.” Although the PedsQL was only administered to children in the study, parents observed their children during both the paper and electronic administrations. Parents were directed to make their ratings and responses about the PedsQL versions based on how they felt the administration method facilitated their child’s completion of the questionnaire. Children and parents were also asked to indicate their preferred method of data collection by circling “pen and paper” or “electronic” on a response form. Again, the parents were directed to provide their preference based on which method they preferred for their child.

Analyses of the reproducibility of PedsQL scores, efficiency, and preference took place following data collection. For our study, each parent gave written informed consent and each child provided assent for participation in the described comparison study. The comparison study was approved by the Education Research and Social & Behavioral Sciences Institutional Review Board at the University of Wisconsin-Madison.

3. Results

3.1. Efficiency

There was not a significant difference in children’s time to complete the PedsQL 4.0 using the paper form or electronic version via a paired t-test (p>.05). See Table 1 for detailed statistics. Time to complete the PedsQL electronically was moderately-to-highly correlated with time to complete the measure using the paper form (Intraclass correlation coefficient = 0.75).

Table I.

Comparison of time to complete the PedsQL 4.0 electronically versus paper based on paired t-test and intraclass correlation.

Paper (mean seconds to complete)	Device (mean seconds to complete)	Mean time difference between Device and Paper (in seconds)	Standard deviation (in seconds)	T statistic	2 tailed p-value (.05>)	Intraclass correlation (Paper time vs. Device time)
170.53	170.47	.06	69.13	.003	1.0	.75

There were no skipped questions or multiple responses on the final implementation of the electronic version of the PedsQL. However, two of the 19 participants skipped or had multiple responses per question using the paper and pencil form. The percentage of subjects with errors using pencil and paper (10.5 %) and without errors using the device was significantly greater than the hypothesized value of 1% (p>.05=.015), via a test of binomial proportion. Nine questions from those two participants were not answered sufficiently with seven question errors out of twenty-three possible questions for one participant and two errors out of twenty-three possible questions for the other. Therefore, efficiency testing revealed that the electronic implementation was as efficient as paper and pencil responding in regards to time to complete the PedsQL, and more efficient in regards to preventing skipped or multiple responses.

3.2 Reproducibility

Difference scores between the paper and electronic scores were calculated for the Physical Functioning (also the Physical Health Summary score), Emotional Functioning, Social Functioning, and School Functioning Scales, the Psychosocial Health Summary (which includes the emotional functioning, social functioning, and school functioning scale scores), and Total PedsQL 4.0 score. No significant differences were found on paired t-tests between any of these scores as delivered by the paper versus electronic version (p>.05) of the PedsQL. Please see Table 2 for complete statistics. Intraclass correlations between the paper versus electronic versions of the scale and summary score differences were all above .7, which prior studies of adult populations have found between well-designed EDC and paper methods (Drummond, Ghosh, Ferguson, Brackenridge, & Tiplady, 1995; Athale, Sturley, Skoczen, Kavanaugh, & Lenert, 2004; Bushnell, Reilly, Galani, Martin, Ricci, Patrick, & McBurney, 2006). Please see Table 3 for ICC values.

Table II.

Results of paired t tests for difference (paper-device) scale, summary, and total PedsQL 4.0 scores

	Mean score difference (paper- device)	Standard deviation	T statistic	2 tailed p value (.05>)
Emotional Functioning	.88	12.9	.28	.78
Physical Functioning/Physical Health Summary Score	.32	7.3	.19	.85
Social Functioning Scale	1.84	10.83	.74	.47
School Functioning Scale	1.312	8.31	.7	.87
Psychosocial Health Summary Score	.29	7.41	.16	.87
Total PedsQL 4.0	.27	6.6	.17	.87

Table III.

Intraclass correlations for scores between paper and electronic forms of the PedsQL 4.0.

PedsQL Scale/Summary Score	ICC (Device*Paper)
Emotional Functioning	.77
Physical Functioning/Health	.86
Social Functioning	.83
School Functioning	.9
Psychosocial Functioning	.91
Total Score	.91

3.3 Acceptability

There were not significant differences in parents’ and children’s visual analog score ratings regarding the ease of reading questions, ease of answering questions, and ease of changing answers between paper versus electronic versions (p>.05) via paired t-tests. Please see Table 4 for detailed statistics. The percentage of children that indicated a preference for using an electronic device (63.2%) over paper was not significantly greater than the hypothesized value of 50% (p>.05 =.36) via a test of binomial proportion. The adult preference for electronic questionnaire delivery (78.9%) to their children was significantly greater than 50% (p < .05=.02).

Table IV.

Comparison between ratings on three parameters of the electronic and paper versions of the PedsQL 4.0 via paired t-test.

	Mean % absolute score difference	Standard deviation	T statistic	2 tailed p value (.05>)
Ease of Reading VAS (Child)	.47	13.5	0.15	.88
Ease of Answering Questions VAS (Child)	8.37	19.2	1.9	.07
Ease of Changing Answers VAS (Child)	11.11	26.2	1.85	.08
Ease of Reading VAS (Parent)	4.89	25.9	0.83	.42
Ease of Answering Questions VAS (Parent)	8.21	22.2	1.6	.13
Ease of Changing Answers VAS (Parent)	15.2	46.95	1.41	.18

4. Discussion

This study sought to determine whether children were able to use an electronic handheld device in an equally efficient and reliable way when compared to the paper standard. It also examined whether children and parents found the electronic delivery of a PRO instrument as acceptable as the traditional paper form. Our results indicate that an electronic implementation of a pediatric QOL questionnaire is equivalent or superior to the standard paper version on all three of these dimensions.

The electronic implementation was more efficient than paper due to the software’s prevention of answering errors and omissions. A significantly larger percentage of children made errors on the paper and pencil implementation of the PedsQL than the electronic version. Time to complete the PedsQL 4.0 by paper and electronically was not significantly different. Additionally, no significant differences were discovered between the paper and electronic versions’ total PedsQL score, physical, emotional, social, and school functioning scales, or physical health and psychosocial summary scores. Additionally, answers between these versions were highly correlated. Finally, no significant differences were found regarding the acceptability of the paper versus software implementation in terms of ease of answering and reading questions or changing answers as perceived by both children and parents. Parents, as a group, significantly preferred the electronic version of the PedsQL to its traditional paper form based on statistical data. No statistically significant difference in preference for PedsQL delivery was indicated by children. Based on the results of our study, we have concluded that using a handheld electronic device for capturing PROs in children is equivalent or better than PRO collection by standard paper forms in a clinical environment.

Delivering the PedsQL and other pediatric PRO measures electronically is likely to decrease answering omissions and errors. Likewise, parent preference for electronic over paper delivery of the PedsQL has implications for parent proxy measures, and suggests that parents may approve of and encourage their children to complete electronic PRO or QOL instruments. Without such approval, electronic administration of pediatric PROs would not be feasible. Additionally, the electronic interface allows for automatic scoring of collected data in contrast to paper; which must be scored manually.

While the use of EDC for PROs has been discussed for at least the last decade (Drummond, Ghosh, Ferguson, Brackenridge, & Tiplady, 1995), EDC has not been well utilized in clinical or home settings. Many issues are typically identified with EDC and need to be addressed during the creation of software interfaces containing PRO measures. Some of these issues include: (1) inadequate presentation of required screen prompts to participants; (2) difficulty with data transmission, whether wireless or via modem line, flashcard, or disc to be mailed to the investigator;(3) need for training and understanding of the software application, which may be challenging for elderly people, children, and the chronically ill; (4) maintaining confidentiality of patient data; and (5) the potential for missing data if participants fail to complete measures as requested. These issues were not addressed in the present feasibility study but will be targeted in our future studies of electronic data collection in children with communication disorders.

Besides the traditional pitfalls of many paper and electronic methods, literacy issues may also contribute to biased data on electronic modes of PRO collection as well as traditional paper versions. When children have low literacy skills, an interviewer often must administer HR-QOL tools; which has the potential of introducing bias into scoring and responses (Mansour, Kotagal, Rose, Ho, Brewer, Roy-Chaudhury, Hornung, Wade, & DeWitt, 2003). Further, low literacy persons (adults and children) may not be included in studies of HR-QOL because they may choose to avoid healthcare situations where such studies are offered due to shame or embarrassment (Hahn & Cella, 2003). Thus, the development of electronic modes of PRO tools that are appropriate for children who are too young to read or have poor literacy skills is necessary. Presenting questions visually on a computer or portable electronic device’s screen in conjunction with auditory delivery of the question is likely to decrease any stigma felt by low literacy participants and/or decrease possible bias (Hahn & Cella, 2003) In the future, we plan to examine an audio component for question delivery for this purpose. Due to the flexibility and ease of use of mobile devices, such as iPad (Apple Computer, Cupertino, CA), video delivery of questions could also be implemented. These multimodality delivery methods that only electronic device can provide are a large advantage over paper methodologies.

A potential limitation of this study is that the electronic collection of PROs via PDA occurred in a clinical setting and therefore our results do not necessarily guarantee that equivalent results would be found in a home or school setting where PDAs might be most useful. Likewise, PDAs when used outside of a clinical setting have the potential to be lost, stolen, or broken, which could ultimately lead to increased research costs and data loss. This limitation will need to be addressed in the future by examining the incidence of lost, stolen, or broken devices and its impact on data loss and research costs.

As previously mentioned, in addition to representing HR-QOL in children who have low literacy skills, the portability of mobile devices allows administration of PROs in a variety of settings. While paper methodologies can be sent out to remote areas, use of PDAs with automatic wireless data transmission capabilities may decrease data loss since paper forms are often not returned by mail. However, future studies will need to determine whether automatic data transmission offsets any data loss that may occur due to lost, stolen, or broken PDAs. Still, automatic data transmission in addition to the host of benefits of electronic collection of PROs via portable devices (i.e. reduction of errors, automatic scoring, potential for multimodality presentation of questions, etc.) may make the electronic collection of PROs of children who are minorities and/or living in poverty especially favorable.

The importance of collecting PROs from such children cannot be overstated, as many have been found to exhibit health disparities when compared with children from the general population (Mansour, Kotagal, Rose, Ho, Brewer, Roy-Chaudhury, Hornung, Wade, & DeWitt, 2003; Chin, Alexander-Young, & Burnet, 2009). For example, HR-QOL is typically higher for healthy children and those with chronic health conditions in the general population when compared with scores of low-income minority children (Varni, Seid, & Kurtin, 2001; Mansour, Kotagal, Rose, Ho, Brewer, Roy-Chaudhury, Hornung, Wade, & DeWitt, 2003). Thus, a more expansive collection of PROs is in order to better address the needs of children across socioeconomic conditions (Mansour, Kotagal, Rose, Ho, Brewer, Roy-Chaudhury, Hornung, Wade, & DeWitt, 2003). Because child health disparities have been documented across different races/ethnicities and socioeconomic status, (Mansour, Kotagal, Rose, Ho, Brewer, Roy-Chaudhury, Hornung, Wade, & DeWitt, 2003; Chin, Alexander-Young, & Burnet, 2009) measuring HR-QOL in children from diverse backgrounds will provide insight into these health disparities, particularly as the percentage of all minorities in the United States is expected to grow significantly over the next 40 years (Stewart & Napoles-Springer, 2000; US Census Bureau, 1996).

While we did not address the possible complications of delivering PDAs to children in remote or disadvantaged areas in the current work, the present research will ultimately serve as the foundation for future projects that will determine the cost of and amount of possible data loss due to lost, damaged, and broken devices in comparison to paper methods. After assessing and attempting to address possible PDA delivery issues for PROs data collection in children, future development and testing of a mobile device application dedicated to the delivery of pediatric PRO measures will be executed.

5. Conclusions

This study was the first step in determining the feasibility of using hand-held electronic devices for the collection of patient reported outcome data from children. Our ultimate goal is to enable efficient, and accurate collection of PROs from typically developing children and those with speech and voice disorders. We plan to develop a method for use in clinics and allow for large-scale delivery to children living in impoverished urban, or remote areas for research purposes. In this preliminary study, we found that use of electronic devices for PRO data collection is feasible when collected in a clinical setting from children with or without communication disorders based on its equivalence with or superiority to specific parameters of paper forms, which represent the current standard. Despite these results, future studies will need to identify and address any major pitfalls of PRO collection via PDA over paper methodogies when delivered to remote or underprivileged areas.

Learning outcomes.

The reader will be able to understand: (1) The potential benefits of using electronic data capture via handheld devices for collecting pediatric patient reported outcomes; (2) The Pediatric Quality of Life Inventory 4.0 is a measure of the perception of general health quality that has distinguished between healthy children and those with chronic health conditions; (3) Past research in communication disorders indicates that voice and speech disorders may impact quality of life in children; (4) Based on preliminary data, electronic collection of patient reported outcomes in children with and without speech/voice disorders is more efficient and equally feasible, reliable, and acceptable when compared to paper forms.

Highlights.

• Responses from 19 children ages 8-12 (9 with speech or voice disorders and 10 typically developing) on both electronic and paper administrations of the PedsQL 4.0 were compared and lead to the findings that there were no significant differences between children’s scores, time to complete the measure, or ratings related to ease of answering questions.

• Parents preferred use of the electronic handheld device for their children, whereas child preference was not significantly greater than the paper form

• Two of the nineteen children made answering errors or omissions on the paper version whereas no children made answering errors with the electronic device.

• Using an electronic device to collect QOL or patient-reported outcomes (PRO) data from children appears to be equally feasible, reliable, and acceptable and more efficient than using a paper form.

Appendix A. Continuing education questions

1. Previous research related to the electronic collection of patient reported outcomes has typically indicated that:

   1. Administering PROs measures to adults electronically is favorable and efficient when compared to standard paper methods.
   2. Administering PROs measures to adults via traditional paper forms is favorable and efficient when compared to electronic data collection.
   3. Children typically prefer PROs measures that are administered by traditional paper forms.
   4. Collecting PROs electronically from children is not feasible.
2. Evidence that speech and/or voice disorders affect children’s lives negatively includes:

   1. Lower scores on the parent proxy of the pediatric voice handicap index by parents with dysphonic children
   2. Negative ratings of audio recordings of dysphonic or dysarthric children
   3. Results from interviews with speech-language pathologists
   4. Results from interviews with children and their parents about their voice or speech disorder
   5. b and d
3. One possible issue with collecting patient reported outcomes via the Internet is:

   1. Older responders may have difficulty using the Internet to access and complete patient reported outcomes
   2. Responders often provide dishonest answers to question items
   3. Responders are typically different from the overall and/or target population
   4. All of the above

4. True or False: Chronically ill children in the general population typically score higher on health related quality of life measures than low income minority children.

5. Based on this study, the use of electronic data capture via a handheld electronic device for collecting patient reported outcomes in children is _____ by parents when compared to paper methods.

   1. Equally accepted
   2. Preferred
   3. Not preferred
   4. None of the above

ANSWER KEY

1. a

2. e

3. c

4. T

5. b