Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Jul 1.
Published in final edited form as: Qual Life Res. 2019 Jan 17;28(7):1793–1801. doi: 10.1007/s11136-019-02109-9

The association of age, literacy and race on completing patient-reported outcome measures in pediatric oncology

Janice S Withycombe 1, Molly McFatrich 2, Laura Pinheiro 3, Pamela S Hinds 4, Frank G Keller 5, Justin N Baker 6, Jenny W Mack 7, Lillian Sung 8, Mia K Waldron 9, Bryce B Reeve 10
PMCID: PMC6579617  NIHMSID: NIHMS1518994  PMID: 30656534

Abstract

Purpose

Age is often used to determine when children can begin completing patient-reported outcome (PRO) instruments or transition to adult instruments. This study’s purpose was to determine relationships between literacy, age, and race and their influence on a child’s ability to understand and complete a PRO instrument.

Methods

The Wide Range Achievement Test was used to evaluate literacy in children and young adults with cancer, participating in a cognitive interview for the Pediatric PRO-CTCAE instrument. 140 participants (7-20 years) were recruited from 8 sites. Logistic regression and multivariable liner regression were used to examine relationships among key variables.

Results

Higher literacy scores were significantly associated with fewer PRO-CTCAE items being identified as “hard to understand” (p=0.017). Literacy scores increased with age, but older participants were more likely to fall behind expected reading levels compared with US norms. A one year increase in age was associated with a 19% increase in the likelihood for being below the expected WRAT word reading score (OR 1.19; 95% CI 1.06-1.33, p=0.003). No associations were found between race and literacy.

Conclusions

Wide variations in literacy were noted across age groups. All participants were able to complete the Pediatric PRO-CTCAE, although most 7 year olds (63%) required reading assistance. Those with lower literacy skills were able to understand items suggesting that multiple factors may be involved in comprehension (developmental stage, concentration, vocabulary, or prior health experiences). Risk for falling below expected literacy levels increased with age implying a need for literacy consideration for cancer patients.

Keywords: literacy, pediatric, patient-reported outcomes, cancer

Introduction

Most children undergoing cancer therapy experience multiple symptoms [1-3]. Among subjective symptoms such as pain or fatigue, documented discrepancies exist between child and proxy and/or child and clinician symptom reports [4; 5]. Accurate symptom reports are necessary to appropriately monitor and manage symptoms in an effort to improve the quality of life for children undergoing cancer therapy. Pediatric patient-reported outcome (PRO) instruments help collect valuable information directly from children but require careful consideration of the factors that may influence data completion and quality, including the child’s cognitive abilities and developmental stage. Many PRO instruments require written questionnaires to be completed independently, which necessitates children/adolescents to have appropriate cognitive skills to read and understand questions and to select answers that match their experiences or perspectives. Emerging literacy skills in children involve a combination of alphabet knowledge, phonological awareness, spelling, and oral language skills encompassing both receptive and expressive vocabulary [6-8]. As such, the ages at which it is feasible to begin PRO data collection in children may vary depending on the type of information being collected and the data collection modality (written, pictorial or verbal).

To self-report, children must have self-awareness; be able to concentrate and pay attention; comprehend instructions, the questions being asked, and the response options; and understand time if questions involve a recall period [9]. PRO instruments, such as visual analog scales, have been successfully administered to children as young as 4 years of age [10; 11]. Children ages 5-7 years, with and without chronic illnesses, have demonstrated the ability to self-report health-related quality of life when questions were administered by an interviewer [12]. Overall, age is used to guide recommendations for when children are able to self-report and when children are able to transition to adolescent or adult versions of PRO instruments.

With growing interest in integrating PRO data into clinical research and healthcare delivery settings to inform decision making, it is critical that we use instruments with maximal ability to elicit the voice of children with chronic illnesses. Although many PRO questionnaires used in pediatrics start around 8 years of age, a greater understanding is needed regarding the extent that a child’s disease may impact their development and cognitive abilities. Among children with cancer, causes for impairments or delays can be pathophysiological in nature (e.g., brain tumor), treatment related (e.g., cranial radiation), or could potentially result from missing school while undergoing prolonged treatment [13; 14]. As such, age alone is an imperfect standard for determining the ability of a child with cancer to complete a PRO instrument.

Large studies, such as the National Assessment of Educational Progress, have documented disparities in academic achievement between white, black and Hispanic children [15]. Over the past 20 years, racial differences in reading achievements have been noted with black and Hispanic students scoring significantly lower than white students [16]. As such, this study included race as a variable to investigate potential interactions with childhood literacy.

The objectives of this study were to assess relationships among literacy, chronological age, and race to determine a child’s ability to understand and complete the Pediatric Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (Pediatric PRO-CTCAE) questionnaire. Using cognitive interview data and literacy assessments, our three specific aims are as follows: 1) determine the association between child literacy level and understanding of the Pediatric PRO-CTCAE questionnaire, 2) determine the association between child literacy level and chronological age to assist with deciding when children can begin completing the Pediatric PRO-CTCAE, and 3) investigate if disparities are present in literacy levels by race/ethnicity.

Methods

This child literacy study was part of a larger study entitled Creating and Validating Child Adverse Event Reporting in Oncology Trials (NIH R01CA175759). The study had Institutional Review Board approval at eight sites: Children’s Healthcare of Atlanta/Emory University, Children’s Hospital Los Angeles, Children’s National Health System, Dana-Farber Cancer Institute/Boston Children’s Hospital, The Hospital for Sick Children, Palmetto Health Children’s Hospital, St. Jude Children’s Research Hospital, and the University of North Carolina at Chapel Hill. The goal of the larger companion study was to develop, refine, and validate the Pediatric PRO-CTCAE and Proxy version of the Pediatric PRO-CTCAE for use in pediatric oncology clinical trials [17].

Eligibility criteria

We included children and adolescents between the ages of 7-20 years, who were English speaking, and actively receiving treatment for any type/stage of cancer. Participants were recruited from in-patient and outpatient treatment settings. Participants ages ≥18 years provided consent for study participation, while children under age 18 had parent/guardian consent, in addition to child assent in accordance with each institution’s policy.

Study design

The Pediatric PRO-CTCAE includes 62 possible symptomatic adverse events that may be subjectively experienced during therapy. One-on-one cognitive interviews were conducted with participants to evaluate their understanding of survey items assessing the symptom’s frequency, severity and interference with daily activities. Data collection was completed during a normal visit for treatment. As the age span for inclusion was diverse, cognitive interviews were grouped to represent distinct developmental stages (7-8, 9-12, 13-15, and 16-20 years). Interviews lasted up to one hour with participants completing the following activities (in order): 1) a paper copy of the PRO-CTCAE measure (Pediatric or Adult version), 2) a cognitive interview, and 3) a word reading activity from the Wide Range Achievement Test version 4 (WRAT) for those participating in the sub-study to access literacy. Digital audio recordings of cognitive interviews and WRAT assessments were obtained with permission from the caregiver and child. After two rounds of cognitive interviews in the larger companion study, 12 additional children (ages 7-9 years) were recruited at Emory University to extend testing of the core symptoms in the Pediatric PRO-CTCAE in the younger age group. Core symptoms were defined as those most commonly occurring across all types of pediatric cancer therapies [18]. Interviewers at all sites completed training related to cognitive interviews and administering WRAT assessments prior to conducting their first interviews. Additionally, interviewers participated in weekly calls to provide feedback on the conduct of interviews and to address any issues. These activities promoted consistency across sites to enhance data quality.

Measures

PRO-CTCAE:

PRO-CTCAE refers to the adult version of the instrument. The Pediatric version of the PRO-CTCAE is a library of items to assess up to 62 symptomatic adverse events (AE). Children and adolescents completed either the PRO-CTCAE or the Pediatric PRO-CTCAE based on age and round of cognitive interview (Fig. 1). Children ages 7-12 years completed the Pediatric PRO-CTCAE instrument. During Round 1, children ages 13-20 years completed the PRO-CTCAE [19] which included 55 original items and 7 new items to match symptoms captured in the pediatric version [17]. In Round 2, 13-15 year olds completed the Pediatric PRO-CTCAE. Instrument development and findings for the qualitative evaluation of the PRO-CTCAE [19-21] and Pediatric PRO-CTCAE [17; 18] have previously been reported. Within the Pediatric PRO-CTCAE library, each AE consists of one to three questions to reflect symptom attributes such as presence, frequency, severity, and/or interference with daily activities.

Fig. 1.

Fig. 1

PRO-CTCAE Instrument by Age Group for Cognitive Interviews

During the first part of the interviews, participants completed the survey measures on their own and marked items as “Hard to Understand” (HTU) if they experienced difficulty with reading or understanding the symptom item or question. If a child could not read the questions independently, the interviewer read the questions to the child. During the interview, field notes were kept to document words or items that children had questions about, but did not explicitly mark as HTU, and/or to document when the child needed help reading all or part of the questionnaire. Additionally, the validity of participants marking items as HTU was confirmed through cognitive interview questions such as “Did you think it was generally easy or hard to answer most of the questions?” and a follow up probe of “What was hard about it?” when appropriate. Questions or terms identified as HTU in round 1 of cognitive interviews were revised, and the new language was evaluated in round 2 of interviews.

Reading Level of PRO-CTCAE Instruments

Flesch-Kincaid grade reading level analyses of the individual questions of the Pediatric PRO-CTCAE were evaluated using Microsoft Word and ranged from 0.1 (less than a first grade reading level) to 11.1(11th grade). Across all 130 questions, the mean grade level was at 4.5 (between a 4th and 5th grade reading equivalency level). Flesch-Kincaid Grade level analyses for the 110 individual questions included in the adult PRO-CTCAE item bank had reading levels ranging from 0.1 to 15.8 (with the higher score translating to an estimated college reading equivalency level). Across all items in the adult PRO-CTCAE bank items, the mean reading level was estimated at a 6.5 grade level.

Wide Range Achievement Test (WRAT) 4:

This study utilized the Word Reading subtest of the WRAT version 4 as a proxy for literacy [22]. The WRAT assesses the basic academic skills of word reading, sentence comprehension, spelling, and math utilizing a norm-reference that was standardized on a sample of 3,000 individuals between the ages of 5-94 years [22]. The Word Reading subtest of the WRAT (henceforth labeled as WRAT-WR) measures letter and word recognition and has two assessments forms (green or blue color coded word cards) which can be interchanged with comparable results [22]. The WRAT-WR subtest includes 70 items (individual letters and words), which children were asked to read out loud. The words are listed in order of increasing phonological complexity. Raw scores for the WRAT-WR subtest range from 0 to 70 as the instrument is scored by giving the participant one point for each letter or word correctly read. The assessment was discontinued when a participant incorrectly read 10 consecutive items or when the child asked to stop as the words became too advanced for them to attempt to pronounce. The internal consistency reliability coefficients for the WRAT-WR subtest range from 0.88-0.98 depending on age group and form version, with a Test-Retest reliability of 0.86 [22]. The WRAT4 also demonstrates an acceptable level of concurrent validity with like measures for Word Reading with a median correlation of .71 [22].

In this study, WRAT-WR subtests were scored by cognitive interviewers at the end of the interview, and raw scores were entered into a study database. After interviews concluded, one or two independent reviewers (JW and MM) verified the WRAT-WR subtest scores using available audio recordings in order to assess scoring accuracy. The raw WRAT-WR subtest scores were transformed to standardized scores using the WRAT4 manual conversion charts for age and word card color [22]. Standardized WRAT-WR scores have a mean of 100 with a 15-point standard deviation [22]. Standardized WRAT-WR scores were categorized using qualitative descriptions and the score ranges provided in the manual [22]. Calculated scores were categorized as “Average/Above Average” if standardized WRAT-WR scores were 90 or greater. Standardized WRAT-WR scores of 89 and below were categorized as “Below Average”.

Statistical Analysis

WRAT-WR scores were examined as continuous and binary outcomes. For the binary outcome, WRAT-WR scores were dichotomized as Average/Above Average vs. Below Average. Using this dichotomized definition of word reading level, t-tests were used to compare mean age (years) and mean child WRAT-WR scores across the two groups. Chi-square tests were used to examine unadjusted differences in cohort (gender, race, ethnicity) and clinical (inpatient/outpatient, cancer type) characteristics by literacy level. To evaluate the association between literacy level and comprehension of the Pediatric PRO-CTCAE, items reported as HTU were counted and grouped (0 HTU, 1 HTU, and 2+). Standardized WRAT-WR scores were utilized only when literacy scores needed to be compared across ages or categorized (under, average or above average) as previously described.

Aim 1: Evaluate the relationship between literacy level and understanding of the PRO-CTCAE questionnaires.

To determine if the number of HTU items was associated with lower WRAT-WR scores, we examined unadjusted associations between the number of HTU items and the child’s raw WRAT-WR scores, using a simple linear regression model with continuous number of HTU items as the outcome. We then estimated multivariable linear regression models adjusting for the child’s age, gender, race, ethnicity, treatment setting (i.e. inpatient or outpatient), and cancer type (leukemia, lymphoma, solid tumor, brain tumor).

Aim 2: Evaluate the relationship between literacy level and chronological age.

Unadjusted associations between literacy level and chronological age were examined using a simple linear regression model with child standardized WRAT-WR scores as the outcome. Standardized scores were used to compare findings across multiple ages. Adjusting for the child’s gender, race, ethnicity, treatment setting, and cancer type, multiple linear regression models were estimated. Additionally, logistic regression models were used to examine associations between chronological age and the dichotomized WRAT-WR scores.

Aim 3: Assess if there are racial/ethnic differences in WRAT scores.

Using chi-square tests, unadjusted differences between a child’s race/ethnicity and their literacy level (below average vs. average/above average) were examined. Multivariable logistic regression models using the dichotomized literacy level variable were estimated, controlling for the child’s age, gender, race, ethnicity, treatment setting, and cancer type. Adjusted odds ratios (AORs) and 95% confidence intervals (CI) were calculated. To determine if the relationship between race/ethnicity and literacy level varied by age, a sensitivity analysis that stratified models for children younger than 13 years and adolescents 13 years and older was conducted. The age of 13 years was selected for stratification to capture the beginning of the adolescent age groups where standardized WRAT-WR scores began to fall below the expected level for age with increased frequency.

Statistical analyses were performed in STATA Version 13.1 with two-sided statistical tests and a significance level of 5%.

Results

Participants

One hundred and forty children participated in the literacy sub-study between February 2014 and December 2016. Five children were excluded from the study as their WRAT assessment sessions ended prior to meeting established stopping rules for the instrument scoring. Participant demographic characteristics for 135 children are summarized in Table 1. The sample was well distributed across ages 7 to 20 years and diverse in demographics (e.g., 47% male, 47% non-white) and cancer type.

Table 1.

Participant Demographics

Characteristic n (%)
Age (in years) 135
   7 15 (11%)
   8 23 (17%)
   9 11 (8%)
   10-15 61 (45%)
   16-20 25 (19%)
Gender
   Female 72 (53%)
Race
   White 72 (53%)
   Black 30 (22%)
   Other 33 (25%)
Hispanic Ethnicity 23 (17%)
Inpatient 65 (48%)
Cancer Type
   Leukemia 65 (48%)
   Lymphoma 29 (21%)
   Solid Tumor 36 (27%)
   Brain Tumor  5 (4%)

Younger children, ages 7 or 8 years, were more likely to request assistance with reading part, or all, of the Pediatric PRO-CTCAE instrument (63% of 7-year olds and 12.5% of 8-year olds). For the children requiring assistance with reading, the average (raw) WRAT-WR score was 27 which loosely equates to a second-grade reading equivalency. Items that equated to higher literacy demand, contained more complex sentence structures, for example questions that asked about interference with normal activities. Individual words included in the Pediatric PRO-CTCAE were generally understandable for most 7-9 year olds after words were changed based on initial cognitive interviews. Children who needed assistance with reading individual words often expressed understanding of the meaning of the word upon hearing the word verbally stated.

WRAT Scores

No notable or statistically significant differences were identified between the original WRAT-WR subtest scores and the verified scores using a sensitivity analysis. Raw WRAT-WR scores ranged from 11-67 on a scale of 0 to 70. Standardized WRAT-WR scores ranged from 65-145 (mean 103; standard deviation 16.5). Table 2 provides a breakdown of standardized scores by age and descriptive score categories.

Table 2.

Standardized WRAT Reading Scores by Age (in Years)

Qualitative
Description
WRAT Score
Range
7 yrs
n (%)
8 yrs
n (%)
9 yrs
n (%)
10-15 yrs
n (%)
16-20 yrs
n (%)
Total
n (%)
Average or Above 11 (73) 22 (96) 10 (91) 50 (82) 13 (52) 106 (79)
 Upper Extreme 130 and up 0 2 0 9 0 11
 Superior 120-129 3 3 0 4 0 10
 Above Average 110-119 2 5 2 9 6 24
 Average  90-109 6 12 8 28 7 61
Below 4 (27) 1 (4) 1 (9) 11 (18) 12 (48) 29 (21)
 Below Average 80-89 1 1 1 10 9 22
 Low 70-79 2 0 0 1 3 6
 Lower Extreme 69 and less 1 0 0 0 0 1
 Total 15 23 11 61 25 135

Aim 1: Literacy Level and Understanding

The majority (87%) of younger children (7-9 years) had standardized WRAT-WR scores that were appropriate for their age. Within this age group, there was one outlier, an 8 year old that displayed an exceptionally advanced reading skill (raw score 59; standardized score 145) which roughly equates to a 12th grade reading equivalency. Similarly, 82% of children 10-15 years had standardized WRAT-WR scores that were average or above average for their age. However, only 52% of participants aged 16-20 years had WRAT-WR scores that were consistent with their chronologic age (Table 2).

When examining the association between literacy scores and understanding, higher raw WRAT-WR scores were significantly associated with fewer HTU items in both unadjusted and adjusted models (p=0.017,p=0.033, respectively). On average, a one-unit increase in raw WRAT-WR scores was significantly associated with a (0.0278) and (0.0342) decline in the number of items marked HTU, in unadjusted and adjusted models respectively. In these models, a decision was made to exclude an outlier (8-year-old child with exceptionally high WRAT-WR score).

Aim 2: Literacy Level and Age

Literacy level and age were highly correlated among participants in our study. There was a statistically significant association between standardized WRAT-WR scores and a child’s age, before (β −0.804, p =0.045) and after adjusting for race, ethnicity, gender, treatment setting, and cancer type (β −0.837, p =0.042). When we examined unadjusted associations between age and the dichotomized literacy level we found a one-year increase in age was significantly associated with a 19% increase in the likelihood of being below the expected (average) WRAT-WR score (OR 1.19; 95% CI 1.06-1.33, p=0.003). Once potential confounders (gender, race, ethnicity, treatment setting and cancer type) were controlled for, we continued to see a statistically significant association between age and likelihood of being below average (aOR 1.23; 95% 1.09-1.37, p=0.001).

Aim 3: Race/ethnicity and Literacy

There were no statistically significant associations between race/ethnicity and being below average for standardized WRAT-WR scores in unadjusted (p=0.823) and adjusted models (p=0.254). In models examining children <13 years of age and adolescents 13 years and older, we did not observe statistically significant associations between race/ethnicity and being below average. These findings were consistent when we examined race as a 3-category variable (White, Black or Other) as well as when we dichotomized race (White vs. Other).

Discussion

Clinical trials include children of all ages making it important to have PRO measures that have been validated for use in children and in compliance with the Food and Drug Administration guidance for PRO measure development [23]. Measures which are valid in children are needed for use in pediatric oncology clinical trials to ensure accuracy of adverse event reporting, but are also needed in the clinical setting to appropriately monitor and treat symptoms with the ultimate goal of improving quality of life indicators. The findings from this study are therefore highly relevant as they provide evidence regarding the influence of literacy on pediatric PRO self-report. Overall, we observed that children with cancer have wide variations in literacy skills across ages. In general, younger children (ages 7-8 years) who needed assistance with reading had WRAT-WR scores that were at or below a second-grade reading level. This finding is consistent with the age ranges presented in a systematic review of instruments validated for use in pediatric oncology showing that a large percentage of instruments begin written data collection around 8 years of age [24]. It has also been noted that the reliability and validity of self-report PROs in children improves around this age [25]. However, age should not be the only indicator of a child’s ability to independently self-report as 63% of 7 year olds and 12.5 % of 8 year olds in our study required interviewer assistance with reading.

Some young children (ages 7-9 years) demonstrated advanced reading skills that were above the reading levels of older adolescents. Lower literacy scores, but not chronologic age, were significantly associated with the number of PRO-CTCAE items that a participant marked as hard-to-understand (HTU). This finding suggests that difficulties in understanding the Pediatric PRO-CTCAE may be influenced by literacy. Other possible explanations for items being marked as HTU may perhaps be related to developmental issues such as concentration and/or self-awareness. Our finding suggests that the Pediatric PRO-CTCAE was able to be completed by children across a wide literacy span. This is important as measurement experts recommend consideration of literacy skills and developmentally appropriate vocabulary during PRO instrument development [25].

Older children (16-20 year olds) in this study were more likely to have standardized WRAT-WR scores below the expected scores for their age. This finding is not unique to adolescents with cancer. In the general population, 63% of 12th graders test below a proficient achievement for their grade level in reading, with 38% scoring at a “below basic” achievement level [26]. Previous studies found that although the rate of growth for reading fluency increases with grade level, the rate of growth decreases with age, especially after second or third grade [27]. However, we found that older children, even those with lower WRAT scores, had little to no trouble independently completing the adult version of the PRO-CTCAE instrument. This observation suggests that a developmental element, in addition to literacy skills, may be associated with reading and interpreting questions related to health status. Vocabulary knowledge increases with age/grade level and these skills are associated with reading comprehension, [8] which may help older children reason out the meaning of more complex questions. It is also plausible that exposure to a treatment intensive illness, like cancer, provided introduction to medical terms and health-related vocabulary which would normally be absent during childhood. An expanded health vocabulary may have assisted with providing a contextual reference for the questions presented in the Pediatric PRO-CTCAE.

Results from multiple rounds of cognitive interviews led to the recommendation that adolescents younger than 18 years of age use the pediatric PRO-CTCAE instrument, but also noted that the adult version of the instrument proved valid and could be utilized, as needed, in adolescents as young as 16 years of age [21]. The findings from our literacy study support these recommendations. Increasing literacy with age may also help to explain why older children (16 years and up) with lower WRAT scores were able to read and respond to the adult version of the PRO-CTCAE instrument which had a higher literacy demand than the Pediatric PRO-CTCAE. A Flesch-Kincaid reading level analysis demonstrated a difference in overall reading level between the Pediatric PRO-CTCAE and the adult PRO-CTCAE instruments, with the adult version scoring an estimated 2 grade equivalency levels higher (mean 6.5 reading level equivalency) and displaying a higher variability in the range of literacy demand among individual questions in the item bank. Older participants, ages 16-20, had higher literacy skills in general (mean 8.1 grade reading equivalency) which supports their ability to transition to the PRO-CTCAE instrument without jeopardizing validity. Cognitive testing also demonstrated the ability of older children (ages 16+) to adequately understand the adult PRO-CTCAE instrument [21].

Prior studies reported decreased cognitive and/or academic abilities among children treated with cranial radiation, [13; 28] especially for those treated at an early age [29]. One study examining children treated for brain tumors reported that children performed better on tests focused on understanding of material (reading comprehension or word comprehension) as opposed to other tests examining spelling, reading speed or basic arithmetic skills [30]. Other studies suggest that chemotherapy alone, especially with intrathecal administration, may be associated with cognitive impairment [31]. Although we did not observe notable differences in WRAT-WR scores by cancer diagnosis, our sample contained relatively few central nervous system (CNS) tumors (n=5) and we did not stratify by +/− cranial radiation.

Our study primarily included childhood leukemia patients. Prior studies in acute lymphoblastic leukemia (ALL) support academic and language outcomes post treatment that are comparable to healthy controls, especially in ALL patients treated on lower (standard) risk protocols without cranial radiation [28; 32; 33]. Our study did not observe any significant difference in literacy level by race/ethnicity (White vs. Blacks vs Hispanics). This is not in agreement with results from other national studies which have found disparities in academic (reading and math) abilities by race/ethnicity in school age children [34; 35]. Some experts argue that socio-economic status accounts for over half of observed differences in racial achievement gaps [36] and should be evaluated, yet socioeconomic status was outside the scope of this study. Other publications have reported differences in academic achievements by race, in healthy children, even when considering economic inequalities [16].The results from our study, however, can be interpreted to mean that race was not a factor in children being able to complete the Pediatric PRO-CTCAE.

Limitations

Several limitations are noted for this study. Bias in study enrollment could have occurred as participants with lower literacy or an aversion to reading may have declined to participate in a study that involved answering questionnaires. Also, the cognitive interviews took between 30-60 minutes so it is possible that the sickest children were excluded as they may have declined to participate in an interview of this length. Additionally, few other studies examining literacy in children actively undergoing treatment for cancer have been conducted and none have reported the WRAT-WR sub-test scores separately. As such, we were unable to compare our findings to previously published results. It is also possible that cognitive effects and decreased academic performance are more common as a late effect of cancer treatment and all of the children participating in this study were fairly early in their therapy. Additionally, the numbers of children in minority racial/ethnic categories were relatively small which may have limited our ability to detect associations with literacy. Lastly, our study examined literacy in relationship to a specific PRO instrument and thus results may differ if repeated using a PRO measure that is more challenging for children to understand. Despite these limitations, our study generated new evidence related to the influence of literacy and age on PRO completion in pediatric oncology patients.

Conclusions

This study offers important insights into the relationship among age, literacy and PRO completion in children with cancer. Our findings underscore the importance of considering literacy regardless of a patient’s age. The exploration of literacy skills in conjunction with cognitive interviews provides additional support for the validation of the Pediatric PRO-CTCAE and its use with pediatric oncology patients. Literacy consideration during instrument development is an important step and should be considered the best practice for fully evaluating the understandability of PRO measures in specific populations. As this study included children (7-20 years) receiving active cancer therapy, it provides valuable information related to changes in literacy with age. Children with less than a second-grade reading equivalence (most 7-year olds) requested assistance with reading some or all of the measurement. Children with emerging reading skills were still able to complete the Pediatric PRO-CTCAE instrument when reading assistance was provided suggesting that comprehension is influenced by vocabulary and prior health experiences. This finding implies that children with lower reading skills may benefit from having an audio version of the instrument available. As PRO measures provide data for use in clinical care (symptom management) and have important implications for drug labeling and toxicity reporting within clinical trials, it is imperative to consider the influence of literacy on child PRO measures which utilize written data collection.

Acknowledgements:

This research was for funded by Alex’s Lemonade Stand Foundation for Childhood Cancer (PI: Withycombe) and by the National Cancer Institute of the National Institutes of Health under award number R01CA175759 (PIs: Reeve and Hinds). The use of REDCap for this project was supported by the Clinical and Translational Science Award program (within the NIH), through Grant Award Number UL1TR002489. The content is the responsibility of the authors and does not necessarily represent the views of Alex’s Lemonade Stand Foundation or the National Institutes of Health.

Footnotes

Conflict of Interest: The authors declare that they have no conflict of interest.

Contributor Information

Janice S. Withycombe, Nell Hodgson Woodruff School of Nursing, Emory University, 1520 Clifton Road, NE, Atlanta, Georgia, 404-727-6936, jwithycombe@emory.edu.

Molly McFatrich, Center for Health Measurement, Department of Population Health Sciences, Duke University School of Medicine, Durham, NC, 919-613-7814, molly.mcfatrich@duke.edu.

Laura Pinheiro, Weill Cornell Medicine, Division of General Internal Medicine, New York, NY, 212-746-7109, lcp2003@med.cornell.edu.

Pamela S. Hinds, Children’s National Health System, 111 Michigan Ave., N.W., Office M7655, Washington, D.C. 20010, 202-476-4432, pshinds@childrensnational.org.

Frank G. Keller, Aflac Cancer and Blood Disorders Center, Children’s Healthcare of Atlanta, 2015 Uppergate Drive, ECC 436, Atlanta, Georgia 30322, 404-785-1112, Frank.keller@choa.org.

Justin N. Baker, Saint Jude Children’s Research Hospital, 262 Danny Thomas Place MS 260, Memphis, TN 38105-3678, justin.baker@stjude.org.

Jenny W. Mack, Dana-Farber/Harvard Cancer Center, 44 Binney Street, Boston, MA 02115, jennifer_mack@dfci.harvard.edu.

Lillian Sung, Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, lillian.sung@sickkids.ca.

Mia K. Waldron, Children’s National Health System, 111 Michigan Ave., N.W., Office M7655, Washington, D.C. 20010, mwaldron@childrensnational.org.

Bryce B. Reeve, Center for Health Measurement, Department of Population Health Sciences, Duke University School of Medicine, Durham, NC, 919-613-7812, Bryce.Reeve@Duke.edu.

References

  • 1.Baggott C, Dodd M, Kennedy C, Marina N, Matthay KK, Cooper BA, & Miaskowski C (2010). Changes in children's reports of symptom occurrence and severity during a course of myelosuppressive chemotherapy. J Pediatr Oncol Nurs, 27(6), 307–315. [DOI] [PubMed] [Google Scholar]
  • 2.Hockenberry MJ, Hooke MC, Gregurich M, McCarthy K, Sambuco G, & Krull K (2010). Symptom clusters in children and adolescents receiving cisplatin, doxorubicin, or ifosfamide. Oncol Nurs Forum, 37(1), E16–27. [DOI] [PubMed] [Google Scholar]
  • 3.Rodgers C, Hooke MC, Ward J, & Linder LA (2016). Symptom Clusters in Children and Adolescents with Cancer. Semin Oncol Nurs, 32(4), 394–404. [DOI] [PubMed] [Google Scholar]
  • 4.Hockenberry MJ, Hinds PS, Barrera P, Bryant R, Adams-McNeill J, Hooke C, Rasco-Baggott C, Patterson-Kelly K, Gattuso JS, & Manteuffel B (2003). Three instruments to assess fatigue in children with cancer: The child, parent and staff perspectives. Journal of Pain and Symptom Management, 25(4), 319–328. [DOI] [PubMed] [Google Scholar]
  • 5.Varni JW, Thissen D, Stucky BD, Liu Y, Magnus B, He J, DeWitt EM, Irwin DE, Lai JS, Amtmann D, & DeWalt DA (2015). Item-level informant discrepancies between children and their parents on the PROMIS((R)) pediatric scales. Qual Life Res, 24(8), 1921–1937. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Whaley WJ, & Kibby MW (1980). Word Synthesis and Beginning Reading-Achievement. Journal of Educational Research, 73(3), 132–138. [Google Scholar]
  • 7.Conrad NJ, Harris N, & Williams J (2013). Individual differences in children's literacy development: the contribution of orthographic knowledge. Reading and Writing, 26(8), 1223–1239. [Google Scholar]
  • 8.Mitchell AM, & Brady SA (2013). The effect of vocabulary knowledge on novel word identification. Ann Dyslexia, 63(3–4), 201–216 [DOI] [PubMed] [Google Scholar]
  • 9.Rebok G, Riley A, Forrest C, Starfield B, Green B, Robertson J, & Tambor E (2001). Elementary school-aged children's reports of their health: a cognitive interviewing study. Qual Life Res, 10(1), 59–70. [DOI] [PubMed] [Google Scholar]
  • 10.Hicks CL, von Baeyer CL, Spafford PA, van Korlaar I, & Goodenough B (2001). The Faces Pain Scale-Revised: toward a common metric in pediatric pain measurement. Pain, 93(2), 173–183. [DOI] [PubMed] [Google Scholar]
  • 11.Dupuis LL, Taddio A, Kerr EN, Kelly A, & MacKeigan L (2006). Development and validation of the pediatric nausea assessment tool for use in children receiving antineoplastic agents. Pharmacotherapy, 26(9), 1221–1231. [DOI] [PubMed] [Google Scholar]
  • 12.Varni JW, Limbers CA, & Burwinkle TM (2007). How young can children reliably and validly self-report their health-related quality of life?: An analysis of 8,591 children across age subgroups with the PedsQL (TM) 4.0 Generic Core Scales. Health and Quality of Life Outcomes, 5(1). Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1769360/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Hutchinson AD, Pfeiffer SM, & Wilson C (2017). Cancer-related cognitive impairment in children. Curr Opin Support Palliat Care, 11(1), 70–75. [DOI] [PubMed] [Google Scholar]
  • 14.Lum A, Wakefield CE, Donnan B, Burns MA, Fardell JE, & Marshall GM (2017). Understanding the school experiences of children and adolescents with serious chronic illness: a systematic meta-review. Child Care Health Dev, 43(5), 645–662. [DOI] [PubMed] [Google Scholar]
  • 15.Reardon SF, Valentino RA, Kalogrides D, Shores KA, & Greenberg EH (2013). Patterns and trends in racial academic chievement gaps among states, 1999–2011. [Google Scholar]
  • 16.Paschall KW, Gershoff ET, & Kuhfeld M (2018). A Two Decade Examination of Historical Race/Ethnicity Disparities in Academic Achievement by Poverty Status. J Youth Adolesc, 47(6), 1164–1177. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Reeve BB, McFatrich M, Pinheiro LC, Weaver MS, Sung L, Withycombe JS, Baker JN, Mack JW, Waldron MK, Gibson D, Tomlinson D, Freyer DR, Mowbray C, Jacobs S, Palma D, Martens CE, Gold SH, Jackson KD, & Hinds PS (2017). Eliciting the child's voice in adverse event reporting in oncology trials: Cognitive interview findings from the Pediatric Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events initiative. Pediatr Blood Cancer, 64(3). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Reeve BB, Withycombe JS, Baker JN, Hooke MC, Lyons JC, Mowbray C, Wang JC, Freyer DR, Joffe S, Sung L, Tomlinson D, Gold SH, & Hinds PS (2013). The first step to integrating the child's voice in adverse event reporting in oncology trials: A content validation study among pediatric oncology clinicians. Pediatric Blood & Cancer, 60(7), 1231–1236. [DOI] [PubMed] [Google Scholar]
  • 19.Basch E, Reeve BB, Mitchell SA, Clauser SB, Minasian LM, Dueck AC, Mendoza TR, Hay J, Atkinson TM, Abernethy AP, Bruner DW, Cleeland CS, Sloan JA, Chilukuri R, Baumgartner P, Denicoff A, St Germain D, O'Mara AM, Chen A, Kelaghan J, Bennett AV, Sit L, Rogak L, Barz A, Paul DB, & Schrag D (2014). Development of the National Cancer Institute's patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Natl Cancer Inst, 106(9). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Dueck AC, Mendoza TR, Mitchell SA, Reeve BB, Castro KM, Rogak LJ, Atkinson TM, Bennett AV, Denicoff AM, O'Mara AM, Li Y, Clauser SB, Bryant DM, Bearden JD 3rd, Gillis TA, Harness JK, Siegel RD, Paul DB, Cleeland CS, Schrag D, Sloan JA, Abernethy AP, Bruner DW, Minasian LM, Basch E, & National Cancer Institute PROCSG (2015). Validity and Reliability of the US National Cancer Institute's Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JAMA Oncol, 1(8), 1051–1059. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Reeve BB, McFatrich M, Pinheiro LC, Freyer DR, Basch EM, Baker JN, Withycombe JS, Sung L, Mack JW, Waldron MK, Mowbray C, Palma D, & Hinds PS (2017). Cognitive Interview-Based Validation of the Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events in Adolescents with Cancer. J Pain Symptom Manage, 53(4), 759–766. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Wilkinson GS, & Robertson GJ (2006). Wide Range Achievement Test-4 (WRAT-4). Lutz, FL: Psychological Assessment Resources. [Google Scholar]
  • 23.Food and Drug Administration. Guidance for industry patient-reported outcome measures: Use in medical product development to support labeling claims. (2009). Retrieved from https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf [DOI] [PMC free article] [PubMed]
  • 24.Pinheiro LC, McFatrich M, Lucas N, Walker JS, Withycombe JS, Hinds PS, Sung L, Tomlinson D, Freyer DR, Mack JW, Baker JN, & Reeve BB (2018). Child and adolescent self-report symptom measurement in pediatric oncology research: a systematic literature review. Qual Life Res, 27(2), 291–319. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Matza LS, Patrick DL, Riley AW, Alexander JJ, Rajmil L, Pleil AM, & Bullinger M (2013). Pediatric patient-reported outcome instruments for research to support medical product labeling: report of the ISPOR PRO good research practices for the assessment of children and adolescents task force. Value Health, 16(4), 461–479. [DOI] [PubMed] [Google Scholar]
  • 26.The Nation's Report Card: 2015. Mathematics & Reading at Grade 12. National Assessment of Educational Progress. [Google Scholar]
  • 27.MacMillan P (2000). Simultaneous measurement of reading growth, gender, and relative-age effects: many-faceted Rasch applied to CBM reading scores. J Appl Meas, 1(4), 393–408. [PubMed] [Google Scholar]
  • 28.Anderson VA, Godber T, Smibert E, Weiskop S, & Ekert H (2000). Cognitive and academic outcome following cranial irradiation and chemotherapy in children: a longitudinal study. Br J Cancer, 82(2), 255–262. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Robinson KE, Kuttesch JF, Champion JE, Andreotti CF, Hipp DW, Bettis A, Barnwell A, & Compas BE (2010). A quantitative meta-analysis of neurocognitive sequelae in survivors of pediatric brain tumors. Pediatr Blood Cancer, 55(3), 525–531. [DOI] [PubMed] [Google Scholar]
  • 30.Lonnerblad M, Lovio R, Berglund E, & van't Hooft I (2017). Affected Aspects Regarding Literacy and Numeracy in Children Treated for Brain Tumors. Journal of Pediatric Oncology Nursing, 34(6), 397–405. [DOI] [PubMed] [Google Scholar]
  • 31.Krull KR, Hockenberry MJ, Miketova P, Carey M, & Moore IM (2013). Chemotherapy-related changes in central nervous system phospholipids and neurocognitive function in childhood acute lymphoblastic leukemia. Leuk Lymphoma, 54(3), 535–540. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Stehbens JA, & Kisker CT (1984). Intelligence and achievement testing in childhood cancer: three years postdiagnosis. J Dev Behav Pediatr, 5(4), 184–188. [PubMed] [Google Scholar]
  • 33.Lewis FM, Perry ML, & Murdoch BE (2013). Longitudinal language outcomes following intrathecal chemotherapy for acute lymphoblastic leukaemia. Int J Speech Lang Pathol, 15(2), 156–164. [DOI] [PubMed] [Google Scholar]
  • 34.Fryer RG, & Levitt SD (2004). Understanding the black-white test score gap in the first two years of school. Review of Economics and Statistics, 86(2), 447–464. [Google Scholar]
  • 35.Clotfelter CT, Ladd HF, & Vigdor JL (2009). The Academic Achievement Gap in Grades 3 to 8. Review of Economics and Statistics, 91(2), 398–419. [Google Scholar]
  • 36.Duncan GJ, & Magnuson KA (2005). Can family socioeconomic resources account for racial and ethnic test score gaps? Future of Children, 15(1), 35–54. [DOI] [PubMed] [Google Scholar]

RESOURCES