Threshold Score for the Self-Report Pediatric Distress Thermometer Rating Scale in Childhood Cancer Patients.

Abstract

Objective:

Although there is enthusiasm for identifying and treating psychosocial problems in childhood cancer patients, there are few validated instruments to help providers identify at-risk children for further assessment. The study objective was to evaluate the sensitivity and specificity of the self-report pediatric Distress Thermometer Rating Scale (Peds DTRS) in childhood cancer survivors and identify a threshold score to help providers classify pediatric patients.

Methods:

We evaluated 54 children 7–17 years old using 178 Peds DTRS longitudinal data points from the cohort that was used for the original pediatric adaptation of the DTRS. We compared Peds DTRS scores against two established standardized measures using a generalized linear mixed model to deal with the dependency in the longitudinal data to estimate ROC curves and related statistics.

Results:

Results indicate that a score of 3 is a reasonable cutoff to identify distress with children 7–17 years old. This cutoff yielded high sensitivity (87.0%) and specificity (79.7%) using the PedsQL Emotional Domain score as the standard. Similar results were obtained using the CDI as the standard, but we are cautious as very few CDI scores reached the cutoff criterion. Exploratory analysis highlighted clinical factors that correlate with increased distress measured using the Peds DTRS.

Conclusions:

The Peds DTRS is a very brief, convenient and rapid screening tool for global distress in children. Further investigation of the Peds DTRS and other tools can improve the ability of providers to prevent and treat the negative emotional consequences of cancer and improve the quality of survivorship.

Background

In a 2008 consensus study report, the Institute of Medicine ¹ asserted that cancer care centers needed to improve their treatment of cancer-associated psychological and social problems, which can limit full recovery. Their report strongly recommended that cancer centers provide supportive care services, including assistance in coping with the emotions accompanying illness and treatment and in managing the disruptions to daily life. In 2015, the American College of Surgeons Commission on Cancer began to require that cancer centers implement screening programs for psychosocial distress as a criterion for accreditation.² The importance of assessment or brief screening in pediatric cancer has been further supported by numerous organizations and prominent groups and a key focus in the 2015 standards of psychosocial care in pediatric cancer.³

The goal of psychosocial distress screening is to refer at-risk patients for further assessment and supportive interventions. By identifying distress early, care providers can intervene to prevent worsening of symptoms or crisis, both of which could negatively impact overall health status. Research has shown that effective integration of distress screening with routine clinical care can allow providers to detect unrecognized problems ^4–6 and improve patient outcomes.^{4, 7, 8, 9}

The National Comprehensive Cancer Network (NCCN) is among the prominent groups that recommend distress screening for all cancer patients.¹⁰ Screening, usually for depression or distress, is increasingly common in adult oncology. The NCCN guidelines support using a Distress Thermometer Rating Scale (DTRS) as a screening tool for rapid assessment of distress among adults. The DT uses a visual analog 10-point scale to rate distress. Although there is evidence that cancer is stressful for children and their families ¹¹, there are few screening tools for pediatric oncology. One tool is the Psychosocial Assessment Tool (PAT) ¹², which is a parent-report screener, and there is a need for validated child-report tools.

More than a decade ago, we adapted the adult DTRS for use with children, in collaboration with an advisory panel of multi-disciplinary psychosocial professionals experienced in pediatric oncology. We developed a child self-report pediatric (Peds DTRS) tool for three age categories combined with a parent/caregiver proxy report tool to further measure the child’s distress. See supplemental materials. We evaluated the measure in a prospective study using 91 patients, their caregivers, and psychosocial support staff, and reported psychometric data supporting its validity and utility.¹³ We later reported the sensitivity of the Peds DTRS in detecting changes in distress among end-of-life pediatric oncology patients and their caregivers.¹⁴

The pediatric distress thermometer has been highlighted as one of two validated screening tools for pediatric oncology ¹¹. Since our initial presentation on the pediatric DTRS ¹⁵, clinicians and researchers from pediatric centers around the world have contacted us to use the Peds DTRS as a brief psychosocial distress screening tool at their facilities. However, there is very little information on the sensitivity and specificity of the Peds DTRS, and there is no established threshold to help clinicians decide when to pursue further assessment.

Wiener and colleagues ¹⁶ investigated the sensitivity of the Peds DTRS in an outpatient pediatric population (ages 7 to 21 years) with mixed diagnoses (cancer, HIV, neurofibromatosis type 1, sickle cell disease, etc.) using 281 caregiver-child dyads. They performed receiver operating characteristic (ROC) analysis to evaluate the sensitivity and specificity of the Peds DTRS using the Children’s Depression Inventory (CDI) as the gold standard in children 7–17 years old. Their results showed that a threshold score of 4 provided good sensitivity (87.5%) and moderate specificity (56.9%), and they reported that potentially raising the cutoff to 5 or 6 improved specificity without substantial loss in sensitivity. In contrast, van der Geest et al. ¹⁷ identified a threshold score of 3 as yielding the best sensitivity (92%) and specificity (79%) among survivors of childhood cancer, 17–44 years old, in an outpatient clinic in the Netherlands.

Our aim was to build upon this work by evaluating the sensitivity and specificity of the self-report Peds DTRS, in a pediatric oncology sample of children with cancer from 7–17 years old, using longitudinal data from the cohort that was used for the original pediatric adaptation of the DTRS. Our goal was to assist providers in selecting a threshold score for further clinical assessment and intervention.

Methods

Participants

The Peds DTRS was originally evaluated in a cohort of 91 English- and Spanish-speaking patients and their parent/caregivers receiving care for pediatric cancer at City of Hope National Medical Center (COH). The study was approved by the Institutional Review Board at COH and informed consent and assent was obtained in accordance with the Declaration of Helsinki and its amendments. Eligibility criteria consisted of the following: ≥ 2 years old, hospitalized in the pediatric unit at COH for cancer or cancer-related complications, and assent given by the patient’s oncologist. For patients under 18 years old, a primary caregiver, usually the mother, was enrolled as a participant along with the child.

Patients completed study questionnaires every 3 months over a 12-month data collection period. All participants were hospitalized at the baseline assessment and most became outpatients by the subsequent assessments. For the current investigation, data for patients who were 7 to 17 years at the time of the baseline assessment were utilized to reduce developmental variation. Data for only those patients with complete responses on Peds DTRS and either of the two gold standards (e.g., CDI) were used in this study (N = 54). The current study included data across five time points (baseline, 3-month follow-up, 6-month follow-up, 9-month follow-up, and 12-month follow-up) to increase the total number of observations available for analyses (e.g., 178 Peds DTRS ratings) and to include distress ratings that were obtained in both inpatient and outpatient settings.

Measures

Pediatric Distress Thermometer Rating Scale (Peds DTRS).

Patients and caregivers completed the Peds DTRS. Participants were instructed to mark a place on the thermometer next to the number (0 to 10) that best represented the level of distress, defined as worry, anxiety, sadness, or fear, on a scale from 0 (no distress) to 10 (severe distress), based on their experience in the previous week. See supplemental materials.

Children’s Depression Inventory (CDI) ¹⁸.

In addition to completing the Peds DTRS, patients completed the CDI, a well-established, widely used 27-item self-report measure of depression for ages 7 to 17 years. For each item, individuals are asked to select the statement that best describes their feelings for the past two weeks, resulting in a composite score. Responses are converted to standardized t-scores with an average score of 50 and standard deviation of 10, with higher scores reflecting more problems. T-scores that fall below 60 indicate typical endorsements for that child’s age and sex; scores between 61 and 64 reflect more concerns than are typically reported; and scores over 65 represent elevated symptoms and trigger definite concern.¹⁸

The Pediatric Quality of Life 4.0 Generic Core Scales – Self and Proxy Report (PedsQL).^19–22

The PedsQL is a 23-item questionnaire that assesses quality-of-life in children 4–17 years old across the following domains: Physical, Emotional, Social, and School Functioning. The Emotional domain has 5 items and was used in this study as a secondary measure of distress. The PedsQL has been normed on 5,193 children/parents and is currently used in various healthcare settings as an established measure of health-related quality-of-life in children.²⁰ There is a child self-report and parent proxy report of the child’s functioning. Both parents/caregivers and children are instructed to identify how much of a problem each item has been during the past month on a 5-point Likert scale, with 0 = never a problem, 1 = almost never a problem, 2 = sometimes a problem, 3 = often a problem, and 4 = almost always a problem. Scale standard scores range from 0 to 100, with higher scores reflecting better wellbeing in the associated domains. The published normative mean score for the PedsQL emotional domain score among healthy children is 79.21 (SD = 18.02); and the normative mean parent proxy is 81.20 (SD = 16.40).²⁰

Statistical methods

The data were analyzed using the R program.²³ Descriptive statistics were used to describe the child and parent/caregiver population, as well as participant ratings and scores on study measures.

ROC analysis was used to evaluate the sensitivity and specificity of the child’s self-reported Peds DTRS distress ratings using the child’s self-report on the CDI and PedsQL Emotional domain as the standard for classifying pediatric patients into distressed versus low or no distress groups. In this analysis, distress was operationally defined as a score at least 1 SD from the normative mean on the CDI (> 60) or the PedsQL emotional domain (< 61).We used 1 SD from the normative mean as a cutoff value to distinguish participants who reported distress levels similar to the top 16% of normative respondents, representing at least moderately elevated scores, compared to those placing below this value. Furthermore, the area under the curve (AUC) that indicates how well a parameter can distinguish between the two groups (i.e. distressed versus low or no distress) was also calculated.

Because the CDI, PedsQL, and Peds DTRS were collected longitudinally across five occasions and the gold standard was binary (e.g., distress vs. low or no distress), a generalized linear mixed model (GLMM)²⁴ with a logit link function (i.e., multilevel logistic regression model with random effects) was employed to estimate probabilities, which were used to construct ROC curves and AUCs.²⁵ A bootstrapping method designed to deals with the dependency in the longitudinal data points was used to calculate the standard errors of AUCs²⁵. The standard errors calculated with 1,000 bootstrapped samples were also used to test the statistical significance of AUCs. Pairwise deletion was used to address the missing data for the estimation of ROC curves, AUCs, and the standard errors of AUCs. Following the recommended procedure for calculating a ROC curve and its AUC on longitudinal data²⁵, we fit two GLMMs, each with a different distress status operationalized from either CDI (CDI Model) or PedsQL (PedsQL Model) as the dependent variable. The fitted models are presented in the Supplementary Table 1 of the online supplemental material. Peds DTRS scores and the time variable for five occasions and the random effects for the intercept were added to both models. Incrementally more complex models with more predictors were built for each GLMM and the best fitting model was chosen in a joint consultation of: (a) a deviance test using a log-likelihood ratio²⁶, (b) Bayesian information criterion (BIC)²⁷ and (c) Akaike information criterion (AIC)²⁸. Randomly varying slopes for time and Peds DTRS were not significant and thus not specified in both GLMMs. The lme4 package²⁹ in R was used to fit GLMMs and the confint function in R was employed to calculate the 95% confidence interval (equivalent to the statistical significance at α = .05) of the coefficients of those GLMMs. After ROC curves and AUCs were calculated, two criteria of Youden’s Index (i.e., sensitivity + specificity - 1) and the distance from the point (0, 1) on the upper left-hand corner of ROC space and any point on the ROC curve were used to select the optimal cutoff for Peds DTRS. The maximum value on the former and the minimum on the latter indicate an optimal cutoff. For our analysis the two criteria agreed and only the Youden’s index values were reported.

Given prior research showing that demographic and clinical factors may predict distress,³⁰ we conducted exploratory analyses to examine correlations between the optimal cut score for child’s self-reported distress and a priori selected demographic and clinical variables: child’s age at self-reported distress rating; family socioeconomic status, maternal education, and race; years since cancer diagnosis, cancer treatment intensity, defined by the presence or absence of bone marrow transplant; cancer type (leukemia or other); and whether the child’s hospitalization at study entry was scheduled or unscheduled, the latter of which were typically due to treatment-related complications. Pearson’s correlations (r) were calculated for continuous variables and Spearman’s rank correlations (r_s) for categorical variables, using p <. 05 for statistical significance.

Results

Patient characteristics

A total of 178 Peds DTRS ratings across five time points were included in the dataset, obtained from 54 children who provided at least one Peds DTRS rating across the five time points. Out of the 51 children who provided Peds DTRS rating at baseline, 41 (80.4%, 3-month), 30 (58.8%, 6-month), 26 (51.0%, 9-month), and 24 (47.1%, 12-month) children provided ratings at the subsequent time points. Those three children who did not provide a rating at baseline participated at later time points (1–3 ratings) except at 9-month. Comparable retention rates were observed for the PedsQL and CDI ratings.

The demographic and clinical characteristics for the 54 pediatric patients who provided these longitudinal assessments are presented in Table 1. Approximately 60% of the children were male, had a mean age of 12.28 years at study entry, with a mean age at cancer diagnosis of 10.17 (SD = 3.78) years old, and mean time since cancer diagnosis of 2.31 years (SD = 2.62) old. Table 1 provides descriptive statistics for other demographics including diagnosis of the participants and their caregivers.

Table 1.

Demographic and clinical characteristics at the time of study enrollment.

Parent/Caregiver Characteristics		Child Characteristics
Caregiver Status (%)		Age (yrs, mean ± SD)	12.28 ± 3.05
Mother	46 (85.2)	Diagnosis Age (yrs, mean ± SD)	10.17 ± 3.78
Father	5 (9.3)	Time Since Diagnosis (yrs, mean ± SD)	2.31 ± 2.62
Other	3 (5.6)	Gender (%)
Caregiver Education (%)		Female	22 (40.7)
Less than HS	10 (17.5)	Male	32 (59.3)
HS Graduate	9 (15.8)	Race (%)
More than HS	25 (43.9)	White	49 (90.7)
CaregiverLanguage† (%)		Non-white	5 (9.3)
English	41 (75.9)	Ethnicity (%)
Spanish	13 (24.1)	Hispanic White	16 (29.6)
SES‡ (%)		Not Hispanic White	33 (61.1)
High	1 (1.9)	Other	5 (9.3)
Middle	38 (70.4)
Low	13 (24.1)
		Diagnosis (%)
		Leukemia	37 (68.5)
		Musculoskeletal Tumor	10 (18.5)
		Other	7 (13.0)
		Bone Marrow Transplant (%)
		Yes	30 (55.6)
		No	24 (44.4)

Note. HS = high school. The total number of children who provided data for the current study is 54.

^†Preferred language in which study forms were completed.

^‡Based on the Hollingshead scale.

Distress ratings

The descriptive statistics (the number of ratings (n), means and standard deviations) of the Peds DTRS, CDI, and PedsQL over five occasions are reported in Table 2. Because we fit two models for CDI and PedsQL separately, the total number of data points for each model was different. Of the total 196 Peds DTRS ratings obtained across the five time points, 18 did not have their concurrent CDI or PedsQL scores and therefore were removed from further analysis, resulting in 178 DTRS ratings. The CDI model included 165 CDI and Peds DTRS concurrent ratings while the PedsQL model included 172 PedsQL and Peds DTRS concurrent ratings.

Table 2.

Descriptive statistics of three distress ratings over time

	Baseline	3-month	6-month	9-month	12-month	Overall
Peds DTRS†(n)	51	42	33	26	26	178
M (SD)	3.61 (2.78)	2.26 (2.41)	2.12 (2.03)	3.58 (3.01)	1.96 (2.72)	2.77 (2.67)
CDI† (n)	44	40	30	26	25	165
M (SD)	44.05 (5.18)	43.08 (7.13)	40.67 (4.97)	42.85 (8.88)	42.96 (10.96)	42.84 (7.37)
PedsQL‡ (n)	50	41	33	25	23	172
M (SD)	66.80 (21.18)	77.07 (16.28)	74.55 (18.17)	74.20 (22.94)	82.39 (22.81)	73.90 (20.50)

Note. Peds DTRS = Pediatric Distress Thermometer Rating Scale; CDI = Children’s Depression Inventory; PedsQL = Pediatric Quality of Life 4.0 Generic Core Scales – Emotional Domain. n = Number of data points included in data analysis for two models; M = mean; SD = standard deviation.

^†Higher scores reflect worse functioning.

^‡Higher scores reflect better functioning.

The overall mean and standard deviation of the Peds DTRS distress ratings among the entire pediatric sample was 2.77 (SD = 2.67). The mean Peds DTRS ratings in the 7 to 10-year-old age group were lower than those of children ages 11 to 17 years (a statistically significant group mean difference; t(133.03) = 2.30, p < .05). Within the 7 to10-year-old group, 43.5% (27 out of 62 ratings) self-reported a distress rating 0; 29.0% (n = 18) of the ratings were 1–3, 24.2% (n = 15) were 4–7, and 3.2% (n = 2) were 10. In contrast, the 11 to17 year-old group (116 ratings total) had a lower frequency of distress scores of 0 (n = 29; 25.0%) and higher rates of distress scores. 37.1% had ratings of 1–3 (n = 43); 25.9% (n = 30) of the ratings were 4–7 and 11.2% (n = 13) were 8–10.

Generalized Linear Mixed Models

The fixed effect of Peds DTRS for the distress standard in both the CDI and the PedsQL models was significant (i.e., logit = 1.74, p < .05 for CDI and logit = 0.63, p < .001 for PedsQL), indicating that a higher rating on Peds DTRS related to a higher probability of distress. However, the fixed effect for the time variable representing the variation over the five occasions was significant (p < .05) only in the CDI Model, indicating the relationship between the distress status by CDI and Peds DTRS varied over the five occasions. This significant longitudinal variation over time was not surprising given that the four CDI scores over 60 (i.e. representing elevated distress) occurred only at either 9- or 12-month follow-up (two ratings at each time point). In contrast, the relationship between Peds DTRS and PedsQL did not vary across time. The fitted models, fit indexes, and additional details about model comparison are presented in the Supplementary Table 1 of the online supplemental material.

Sensitivity of the Peds DTRS relative to CDI and PedsQL

Figure 1 displays two ROC curves estimated. The estimated AUC was .997 using the CDI as the criterion measure. This result indicates that the Peds DTRS can accurately distinguish distressed from non-distressed children nearly 100% of the time. However, the bootstrap procedure for the AUC standard error calculation did not work because only four data points out of 165 who completed both CDI and Peds DTRS met the criteria for elevated distress (i.e., CDI > 60). As a result, many bootstrap samples consisted entirely of non-distress data points (i.e., CDI ≤ 60) and AUCs were not calculated. The 4 CDI cases that met the cut off criteria had the following t-scores: 71, 66, 84, 63, with Peds DTRS ratings as follows: 8, 10,10,6.

Figure 1.

Receiver operator characteristics (ROC) curves. Orange-Solid Line: ROC curve for the Pediatric Distress Thermometer Rating Scale (Peds DTRS) compared with the Children’s Depression Inventory (CDI); AUC = .997. Green-Dotted Line: ROC curve for the Peds DTRS compared with the Pediatric Quality-of-Life Inventory (PedsQL) Emotional domain; AUC = .917, p < .001, 95% C.I. [.852, .982].

The estimated AUC using the PedsQL as the gold standard was .917 (p < .001, 95% C.I. [.852, .982]), indicating that the Peds DTRS can accurately distinguish distressed from non-distressed children 92% of the time. To verify the stability of the estimated AUC standard error and the 95% CI, we increased the bootstrap sample from 1,000 to 5,000. The differences between the estimates were negligible, signifying the stability of the bootstrap estimates.

Table 3 presents the specificity and sensitivity for each threshold score compared to the criterion measures in the ROC analysis. When the CDI scores were used as the standard, the Youden’s index indicated a cutoff score of 3 as the optimal threshold, yielding a sensitivity of 100.0% and specificity of 99.4%. These values suggest that this threshold score identifies a child with distress 100.0% of the time (true positive rate) and identifies children without distress 99.4% of the time (true negative rate). A score of 1 or 2 has the same sensitivity but slightly lower specificity. However, these results are viewed as unreliable given the extremely low proportion (only four in 165) of the distress status identified using the CDI (> 60), which reasonably affects the estimated sensitivity and specificity. Interestingly, when the PedsQL was used as the standard, the same score of 3 yielded a sensitivity of .870 and specificity of .797(54 out of 172 (31.4%) data points met the criteria for elevated distress). A score of 4 yielded a slightly worse sensitivity, .759 and a better specificity, .898, but a score of 3 achieved a slightly higher value on the Youden’s Index.

Table 3.

Sensitivity and specificity of Peds DTRS thresholds compared to child self-reported CDI and PedsQL Emotional domain scores.

	Child Self-Report CDI			Child Self-Report PedsQL Emotional
Threshold	Sensitivity	Specificity	Youden	Sensitivity	Specificity	Youden
0	1.000	0.000	0.000	1.000	0.000	0.000
1	1.000	0.981	0.981	0.981	0.483	0.465
2	1.000	0.981	0.981	0.926	0.686	0.612
3	1.000	0.988	0.988	0.870	0.797	0.667
4	0.750	0.994	0.744	0.759	0.898	0.658
5	0.500	0.994	0.494	0.667	0.932	0.599
6	0.500	0.994	0.494	0.556	0.958	0.513
7	0.500	1.000	0.500	0.370	0.983	0.353
8	0.500	1.000	0.500	0.278	1.000	0.278
9	0.250	1.000	0.250	0.167	1.000	0.167
10	0.000	1.000	0.000	0.000	1.000	0.000

Note. Youden = Youden’s J Statistic or Youden’s Index (Sensitivity + Specificity – 1). The maximum value on the Youden’s Index indicates an optimal threshold value. The optimal cutoff values are highlighted (in bold and underlined).

Based on the sensitivity-specificity results, we determined that a cut off score of 3 was best supported as the optimal clinical threshold on Peds DTRS. The two separate standards of CDI and PedsQL concurrently supported this score although the sensitivity and specificity from the former was nearly perfect, which we ascribe to the restricted data distribution of CDI scores in the sample (i.e., only four data points out of 165 assigned to the distress level). Although a score of 4 using PedsQL used as the criterion achieved similar high sensitivity and specificity; the Youden’s index for a score of 3 was slightly higher and we further reasoned that it is better to identify all children who are distressed, even at a slightly increased risk of including false positive case.

Factors correlated with a Peds DTRS of ≥ 3

As an exploratory analysis, we evaluated the relationship between clinical and demographic factors and a Peds DTRS score ≥ 3. Child’s age at assessment (r = .23, p = .001) was positively correlated with a Peds DTRS score ≥ 3, such that the older the child, the greater the distress level. Having an unscheduled or unplanned hospitalization (r_s = .18, p < .05), and receiving a bone marrow transplant were both correlated with a Peds DTRS score ≥ 3 (r_s = .21, p < .01) or higher distress, while having a leukemia diagnosis was associated with lower distress relative to non-leukemia cancer diagnosis (r_s = −.21, p < .01). The time (years) since cancer diagnosis was negatively correlated with a Peds DTRS score ≥ 3 (r = −.18, p < .01), such that the longer time since the initial cancer diagnosis, the lower the distress level. The demographic factors of socio-economic status (SES), race, and parent education were not significantly correlated with the Peds DTRS distress level, while the child’s age at cancer diagnosis (r = .28, p < .001) was correlated such that children with older age at diagnosis were more likely to have a Peds DTRS > 3.

DISCUSSION

This purpose of this study was to propose a threshold score on the Peds DTRS, in a pediatric oncology population, to identify children for further assessment. ROC analysis of 178 Peds DTRS ratings from 54 youths (7–17 years old) revealed that a threshold score of 3 is a reasonable cutoff for the Peds DTRS when compared to either the CDI or the PedsQL Emotional domain. We employed GLMMs to model dependencies in the longitudinal ratings data of the Peds DTRS, CDI, and PedsQL collected over five occasions throughout a 12-month period. Exploratory analysis indicated that multiple clinical factors correlate with a higher Peds DTRS distress level, including having an unplanned hospitalization, bone marrow transplant (which typically involves longer hospitalization), and less time since cancer diagnosis. Overall, this work provides a guideline for measuring distress using the Peds DTRS and indicates important factors that influence distress and can be followed up in future studies.

Although we initially selected the CDI as the gold standard measure against which to compare the Peds DTRS, only four data points of our sample had CDI scores > 60, indicating that even mild depressive symptoms were rare in this sample of childhood cancer survivors. In contrast, 54 out of 172 (31.4%) met the criteria for distress (< 61) on the child reported PedsQL Emotional domain. Given this distribution, the ROC analysis relative to the PedsQL was viewed as more informative than the ROC analysis using the CDI as a criterion. The finding of a low incidence of depression on the CDI, an established pediatric measure of depression, is consistent with published reports of relatively good adjustment and low levels of mood disturbance among pediatric cancer survivors, ³¹ with levels of psychological distress (e.g., anxiety, depression) similar to those of healthy peers.^{30, 32}

Relatedly, future research should further investigate the very low distress rates measured in younger children as 43.5% of the Peds DTRS ratings among the 7–10 year old group in our sample were 0; in contrast, only 25.0% of ratings in the older age group (11–17 years old) had this value. It is not clear whether younger children truly have very low distress, or if different approaches are needed to more reliably ascertain their self-report as young children tend to have a here-and-now focus where distress is low if their needs and symptoms are well-managed at the time of assessment, even if they felt upset the prior day. Nevertheless, our data suggests that clinicians should consider the child’s age in decision making about which sources of information to include in the clinical assessment. For younger children, self-report may be augmented by proxy parent-report about the child’s functioning, which typically integrates a longer-term perspective. However, obtaining the self-reports from the children remains important as parents’ own distress levels, or other related factors, could influence parental reports of child distress.

Our results show that the self-report Peds DTRS can distinguish distressed patients from those with no or low distress, using either the PedsQL or the CDI as the criterion. Although we recommend a threshold score of 3 as a classifier, a more nuanced interpretation may be warranted in certain situations. Sensitivity and Specificity are inversely proportional to each other, so when sensitivity increases, specificity decreases. For detecting distress in children, some clinicians may prefer higher sensitivity over specificity, to identify all or most of the distressed patients, at the risk of falsely identifying those who are not truly distressed. For settings with limited resources, a clinician might prefer to conserve follow-up resources only for those who are high risk, in which case a higher specificity would be preferred. Our results for the Peds DTRS are consistent with recommendations of 3 or 4 as a threshold score for the NCCN distress thermometer in adult oncology. ^34–35

Study limitations

The present study has several limitations. First, the sample and number of ratings used did not allow for a detailed examination of the impact of age on self-reported distress levels. For example, it is possible that the threshold for older adolescents should be different than the threshold for younger children. In the general population, the rates of depression and anxiety tend to be low during childhood but increase substantially during adolescence and young adulthood.³³ Second, the sample in this study was recruited from a single site and therefore findings need to be replicated using samples from other pediatric cancer treatment sites and regions. It is also possible that the ROC analyses results would have yielded different results if we had used different criterion measures. Lastly, the number of CDI scores over 60 in the range of distress was only four in this study, thus limiting the stability and interpretability of the generalized linear mixed model that used the CDI as the gold standard. Similar future studies with more CDI data points in the distress level would allow more stable findings on optimal threshold values. Future studies should also confirm the cut off score with a sample that is closer to diagnosis and potentially more likely to be exhibiting distress.

Clinical Implications

Findings from the current study indicated that a threshold score of 3 yielded the most optimal sensitivity and specificity, making it the best choice to minimize the number of children with distress who go undetected. We note that a higher threshold score is advised for clinicians who need to minimize the number of non-distressed patients who would be misidentified as distressed. Individuals who meet the threshold criteria would then be assessed further, ideally with a clinical interview, to triage for appropriate levels of intervention services, if indicated.

Conclusions

The Peds DTRS is a very brief, convenient and rapid screening tool to evaluate global distress in children. Similar to the adult distress thermometer, it is attractive because of its time efficiency. Indeed, the Peds DTRS has now been broadly disseminated via many requests from providers at national and international pediatric clinics; therefore, we strongly encourage further research to better understand its benefits and limitations as a screening tool in the clinical setting.

DATA AVAILABILITY STATEMENT:

The data that support the findings of this study are available from the corresponding author upon reasonable request