Pediatric Patient and Caregiver Agreement on Perioperative Expectations and Self-Reported Outcomes

Abstract

Introduction:

Alignment between pediatric patients and caregiver perspectives on patient-reported outcome (PRO) data is contingent upon context. We aimed to assess agreement between patient and caregiver responses to a series of perioperative domains.

Methods:

Agreement between pediatric patients and caregiver responses to preoperative and postoperative surveys about surgery preparedness, perioperative expectations, PRO Measurement Information System (PROMIS) measures for overall health and pain, and reaching milestones gathered as part of an ongoing clinical trial for children undergoing gastrointestinal surgery, was evaluated. Gwet’s AC and Spearman’s correlation coefficients were calculated, as appropriate, to assess agreement.

Results:

Of 209 enrolled patients, 65 (31.1%) dyads completed all three surveys and were included. For the domains of education, expectations, and comprehension, patients and caregivers had good agreement with Gwet AC₁ with values of 0.80, 0.61, and 0.64, respectively. For milestones, patients and caregivers had very good agreement (Gwet AC₁ of 0.95). Milestones measured whether patients achieved certain goals within a prespecified time, including enteral intake (Gwet AC₁ 0.91 and 0.92 respectively), transition to oral pain medication (Gwet AC₁ 0.94), ambulation (Gwet AC₁ 1.00), and return of bowel function (Gwet AC₁ 0.97). There was moderate to strong agreement between patients and caregivers on PROMIS pain questions (Spearman’s correlation: 0.71 preoperatively and 0.51 postoperatively). On PROMIS global health questions, there was strong agreement (0.69 preoperatively and 0.65 postoperatively).

Conclusions:

Pediatric patient and caregiver agreement on perioperative survey items ranged from moderate to strong. Caregivers’ responses may be acceptable when some patient-level responses are not available.

Introduction

Many performance measures for physicians and hospital systems now include patient-reported outcomes (PROs) in addition to clinical outcomes.^1,2 In 2004, the National Institutes of Health (NIH) started the patient-reported outcomes measurement information system (PROMIS) to provide clinicians and researchers with robust, standardized, patient-reported measures for both pediatric and adult patients. These measures can be used for a range of conditions and diseases and the scores can be compared across groups of individuals.³ The PROMIS pediatric measures were specifically designed for children and adolescents between the ages of 8 and 17 y so children can reliably be surveyed about their experiences with chronic conditions and pain.^3,4

Collecting PRO data among pediatric populations can be challenging, so parents/caregivers are often enlisted as proxies for reporting. The PROMIS parent proxy measures were designed for this purpose.^5,6 Despite having validated PROMIS measures for pediatric patients and their caregiver proxies, alignment between pediatric patient and caregiver perspectives can vary depending on the topic. Caregivers and children have been continually shown to have discrepancies in their ratings of pain intensity and frequency.^7–11 However, caregivers and children have a higher degree of agreement about physical health.^4,12 Given the variation in pediatric patient and caregiver agreement, it is unclear when caregiver proxy reports are acceptable for pediatric patients.

This study uses data from the ongoing multicenter, a prospective study entitled ENhanced Recovery In CHildren Undergoing Surgery (ENRICH-US). The goal of the study is to assess agreement between pediatric parents and their caregivers. ENRICH-US is a study designed to evaluate the implementation and effectiveness of a newly developed, 21-element, enhanced recovery protocol for children undergoing elective gastrointestinal surgery. As a part of the study, both patients and their caregivers are asked to complete several surveys, including a preoperative survey and two postoperative surveys related to perioperative education, expectations, comprehension, milestones, and PROMIS measures.

Methods

Study setting

Data were collected from patients and caregivers participating in the ENRICH-US study.¹³ Pediatric patients, from 10 to 18 y of age, undergoing elective gastrointestinal surgery at 18 US pediatric surgical centers, and their caregivers, as proxies, were included. The trial protocol (Protocol #0003920) was approved by Advarra, Inc (Columbia, MD) which serves as the central institutional review board for all study sites. Study team members at each site, screened, recruited, and consented eligible subjects. The consent process included parental consent (unless the subject was ≥18 y old) and child assent (for subjects <18 y old). Caregivers/patients had to be English or Spanish speaking.

Data collection

The ENRICH-US clinical trial includes 1 preoperative and two postoperative surveys for both the children and their caregivers. The surveys were sent via email to the patients and caregivers. To assist with survey completion, clinicians and research staff could also administer the surveys to the patients and their caregivers in person. The preoperative survey could be administered in the preoperative clinic or the operating room holding area. The first postoperative survey could be administered prior to hospital discharge and the second postoperative survey at the follow-up clinic visit within 30 d. The preoperative survey covers topics related to preparedness for surgery and whether the care team explained the operation, perioperative expectations, and PROMIS measures related to overall health and pain. The postoperative surveys covered topics related to understanding the operative and perioperative processes, when and what the patient could eat, when the patient got out of bed and had return of bowel function, and PROMIS measures related to overall health and pain. The survey questions focused on four main domains: education, expectations, comprehension, and milestones. The education domain was comprised of questions related to receiving information on how to prepare for surgery and what would occur during the operation itself. The expectations section had questions related to recovering after surgery. Comprehension questions asked if patients and their caregivers understood the steps of their surgery and recovery. The milestone domain asked if patients met goals related to enteral nutrition, oral pain medications, getting out of bed, voiding, and having bowel movements after surgery (Supplemental Table 1). The PROMIS measures related to pre and postoperative global health and pain were reported separately from these four domains (Supplemental Documents 1-4).

Study cohort

The data were collected from pediatric patients, aged from 10 to 18 y, who underwent surgery between July 1, 2020, and July 1, 2021, and their caregivers, participating in the ongoing ENRICH-US clinical trial. For responses to be included in the analysis, both the child and their caregiver had to have filled out the surveys completely.

Statistical analysis

To assess agreement between patients and their caregivers, Gwet’s AC₁ correlation coefficients were calculated for each of the questions from the surveys and the average of these were reported for each of the four domains. Gwet’s AC₁ is used to measure inter-rate agreement for categorical assessments and is the statistic of choice for two raters. Interpretation of Gwet’s AC₁ is as follows: poor agreement is less than 0.2, fair is 0.21 to 0.4, moderate agreement is 0.41 to 0.6, good is 0.61 to 0.8, and very good is 0.81 to 1.^14,15 Spearman’s correlation coefficients were calculated for the PROMIS measures since continuous T-scores are generated from these questions. Interpretation of Spearman’s correlation coefficient is as follows: very weak 0-0.19, weak 0.20-0.39, moderate 0.40-0.59, strong 0.60-0.79, and very strong 0.80-1.0.¹⁶ Paired t-test with mean and Wilcoxon matched-pairs singed-ranks tests with median Likert scale values were used as appropriate to evaluate directionality of agreement between patients and caregivers. Significance level was set at 0.05. All analyses were performed using STATA version 17.0 (College Station, TX).

Results

During the study period, 209 pediatric patients underwent surgery and 196 (93.8%) patients and/or caregivers filled out at least one survey. There were 141 (67.5%) dyads of patients and caregivers who each filled out at least one of the same surveys, and 65 (31.1%) dyads of patients and caregivers who completed all three surveys were included in the study. Of the 65 patients, 44 (67.7%) were male, 52 (80%) were White, 34 (52.3%) had private insurance, and 47 (72.3%) underwent surgery for inflammatory bowel disease (Table).

Table 1.

Patient demographics for the 65 patients included in the study.

Variable	N	%
Gender
Male	44	67.7
Female	21	32.3
Race
White	52	80
Black	11	16.9
More than one race	1	1.5
Other	1	1.5
Ethnicity
Not Hispanic or Latino	56	86.2
Hispanic or Latino	6	9.2
Unknown	3	4.6
Insurance status
Private	34	52.3
Public	29	44.6
Other	2	3.1
Indication for GI surgery
Inflammatory bowel disease	47	72.3
History of trauma	3	4.6
History of prior medical condition requiring bowel surgery	7	10.8
History of cancer	1	1.5
Familial adenomatous polyposis	5	7.7
Other	2	3.1

Education, expectations, comprehension, and milestones domains

There was good agreement (Gwet AC₁ 0.80) in the education domain, which included questions about receiving general preoperative information, how to prepare for surgery, and information about the operation itself. Within this domain, there was very good agreement between patients and caregivers on receiving information about the operation itself (Gwet AC₁ 0.83); good agreement for expectations (Gwet AC₁ 0.61), which included questions related to how to recover from surgery and control pain. Within this domain, questions related to how to recover from surgery had good agreement (Gwet AC₁ 0.72), while questions related to pain control only had moderate agreement (Gwet AC₁ 0.49). Patients and caregivers reported similar agreement on receiving information about how to recover from surgery, median Likert score 3 (interquartile range [IQR]: 3-3) versus 3 (IQR: 2-3), respectively (P = 0.13). Patients reported slightly higher agreement on receiving information related to pain control after surgery more frequently than their caregivers, median Likert score 3 (IQR: 2-3) versus 3 (IQR: 1-3), respectively (P = 0.12). Overall, comprehension about the operation had good agreement (Gwet AC₁ 0.64), which included questions related to understanding the steps of the operation, recovery, and when the patient could start eating and go home after surgery. Within this domain, understanding the steps of the operation had very good agreement (Gwet AC₁ 0.83), but understanding when the patient could start eating had moderate agreement (Gwet AC₁ 0.46). Fewer patients (n = 30, 46.2%) than caregivers (n = 37, 56.9%) thought they would be able to eat the same day as surgery P = 0.06. For milestones, patients and caregivers, overall, had very good agreement with a Gwet AC₁ of 0.95. Milestones measured whether patients achieved certain goals within a prespecified time, including when and what patients ate after surgery (Gwet AC₁ 0.91 and 0.92, respectively), transition to oral pain medication from intravenous (Gwet AC₁ 0.94), getting out of bed (Gwet AC₁ 1.00), and return of bowel function (Gwet AC₁ 0.97).

PROMIS measures

PROMIS measures were evaluated with Spearman’s correlation coefficient and demonstrated strong agreement between patients and caregivers on PROMIS pain questions preoperatively (0.71) and only moderate agreement, postoperatively (0.51). Caregivers reported higher average T-scores for pain both pre (55.3 versus 53.0, P = 0.03) and postoperatively (54.8 versus 50.6, P < 0.01) than the patients. On PROMIS global health questions, Spearman’s correlation demonstrated patients and caregivers had strong agreement both pre (0.69) and postoperatively (0.65, Fig.). Caregivers reported lower average T-scores for global health both pre (42.3 versus 43.0, P = 0.49) and postoperatively (43.0 versus 46.4, P < 0.01).

Fig 1.

This figure shows the agreement between patients and their caregivers on perioperative education, expectations, comprehension, milestones, and PROMIS measures. *All calculations were done with Gwet’s AC1 test of agreement except those with an Asterix which were performed with a Spearman’s correlation coefficient.

Discussion

PROs are increasingly being used to evaluate quality of care among hospital systems. For pediatric populations, caregivers, as proxies, often provide feedback on behalf of their children as it can be difficult to collect PRO data from a pediatric population directly.^5,6 Studies about agreement between pediatric patients and their caregivers show a wide variation.^4,7–12 This study found good agreement between pediatric patients and their caregivers on questions about perioperative education, expectations, and comprehension, and very good agreement on reporting achievement of perioperative milestones. With regard to PROMIS pain measures, patients and their caregivers had good agreement preoperatively, but only moderate agreement postoperatively, for the global health questions they had good agreement both pre and postoperatively. In our study, caregivers tended to overestimate the patients’ pain and underestimate their overall global health. Overall, this study suggests that caregivers’ proxy responses may be acceptable for pediatric patients undergoing surgery when patient-reported data are not obtainable.

In the domains of education and comprehension, the questions related to the operation itself and understanding the steps of surgery had the strongest agreement, suggesting that patients and their caregivers are likely receiving the same information at the same time from clinicians, as well as, repeatedly, such as in the preoperative clinic and during the consent process. Additionally, patients and caregivers can find information about surgical procedures relatively easily online. Agreement on education and comprehension of the postoperative recovery process was good, but not as strong as the steps of the operation. This is likely because information about the postoperative recovery process is less available online and providers may vary in the information provided.

The milestone domain had the highest agreement which is likely because the questions ask about easily observable outcomes, including when the patient was able to eat, get out of bed, transition from intravenous to oral pain medication, have a bowel movement, and urinate.

Finally, the PROMIS pain measures had better agreement preoperatively than postoperatively which suggests that caregiver proxy reports about pain are not as good. Agreement of PROMIS pain measures was significantly poorer, postoperatively, which indicates that caregiver proxy report of acute pain is unreliable. This finding is consistent with many prior studies that have shown discrepancies rating pain intensity and frequency between pediatric patients and their caregivers.^7–11 Chambers et al. found that poor to fair agreement between parents and their children when rating pain with parents underrating their child’s pain on the day of surgery and on postoperative day 1, but not on postoperative day 2.⁸ Another Chamber’s study that looked at five different face scales for rating pain found poor agreement between parents and children, but this study found that parents had significantly higher pain ratings than their children on all five scales.⁷ Palermo et al. found that parents and children had poor agreement on pain intensity, fair agreement on pain frequency, and moderate agreement on pain duration. They found greater parent-child discrepancy about pain frequency when parents perceived that their child had fewer functional limitations and when the children reported depressive symptoms.¹⁰ Finally, Garcia-Munitis et al. found poor to moderate agreement in rating present pain between parents and children. The results were similar to the second Chambers article reporting that children reported lower pain scores than their parents.⁹ Not only are there multiple studies showing that there are discrepancies in rating pain between children and parents, but that parents both over and underestimate their child’s pain. This suggests that, ideally, we should collect patient-reported responses related to pain directly from the patient rather than the caregiver. More research is needed to better understand what circumstances and factors may influence when caregiver proxies over or underestimate their children’s pain, to better interpret caregiver proxy responses and adequately treat the children’s’ pain.

Limitations

There are several limitations to this study. Only 31% of pediatric patient and their caregiver dyads filled out all surveys, resulting in a modest sample size, despite many attempts to improve the survey response rate, such as having patients and their caregivers complete the first postoperative survey prior to discharge from the hospital, and provide them the opportunity to fill out the preoperative and second postoperative surveys during a clinic visit. Another method for improving survey response rate could be requiring completion prior to being seen by a clinician. For example, inpatient advanced practice providers could require completion prior to providing discharge papers. We are unable to assess whether patients and their caregivers filled out the surveys together or if caregivers inquired of patients about any of the questions when completing the survey. In addition, we cannot account for whether the same caregiver filled out each of the surveys or if more than 1 caregiver (e.g., mother for one survey and father for another) completed the surveys. However, Garcia-Munitis et al. assessed agreement about caregiver proxy report pain and disability between mothers and fathers and found very similar responses.⁹ In the future, independence of responses could be monitored and encouraged if surveys are completed in the clinical setting. Finally, we did not evaluate child or caregiver depression, which has been shown to be associated with disagreement.¹⁰ We also did not assess psychosocial issues like stress, family pain history, or family environment, which also can affect how pain is perceived.^17,18 Future surveys could include questions about depression, family environment and stress, and family pain history.

Conclusions

Pediatric patients and their caregivers have good agreement on survey items about perioperative education, expectations, and comprehension, and very good agreement on meeting milestones. Preoperatively they have good agreement on PROMIS pain measures, but only moderate agreement postoperatively. It is clear that patient and caregiver agreement varies by domain, topic, and time period of the survey item. In the pediatric surgical population, ideally PROs should be reported directly by the patient, but if patient-reported responses are not possible, it may be acceptable to use caregivers’ responses for some domains and topics. For example, proxy-reported postoperative pain should be interpreted with caution.