Level of Agreement in Perceived Improvement in a Cohort of Parent–Child Dyads Following an Interdisciplinary Pain Management Program

ABSTRACT

This study assessed the level of agreement of perceived improvements among pediatric patients with chronic pain and their parents following an interdisciplinary pain management program. This is a retrospective chart review of a heterogeneous group of pediatric patients with chronic pain who completed an interdisciplinary pain management program (IPMP) and their parents (N = 172 pairs). The IPMP took place in an outpatient pain management center affiliated with an academic rehabilitation hospital. The outcome measures included the Multidimensional Patient Impression of Change (MPIC) and the MPIC parent version were administered post‐treatment. The levels of agreement on the MPIC were analyzed between child and parent using the free‐marginal kappa statistic. The majority of both the parents and the children evaluated the patients to be much or very much improved in six of the eight domains assessed. The levels of agreement in the ratings were fair to moderate among the patient–parent dyads. Parents endorsed significantly higher levels of improved overall pain and mood than their child did, whereas the patients reported greater levels of improved ability to cope with pain than their parents did. Both the patients and their parents reported high levels of perceived improvement following the IPMP. However, there were some differences in their ratings of improvements in overall pain, mood, and ability to cope with pain. Given that some of the perceptions were discrepant, it is recommended that feedback be obtained from the patient and their parent.

1. Introduction

Pediatric chronic pain is a common costly diagnosis characterized by pain that persists or recurs for more than 3 months in people < 20 years old regardless of if an identifiable underlying cause is present and is associated with high healthcare utilization [1, 2, 3]. The condition is commonly accompanied by changes in lifestyle, personality, functional status and symptoms of depression and anxiety [4]. The estimated global prevalence rate is as high as 20% for pediatrics with any type of chronic pain or general pain, and it is estimated to cost Americans $19.5 billion annually [5, 6]. Children and adolescents often rely on their parents or caregivers to initiate and navigate the treatment process; however, they may not always share the same perspective with respect to important treatment‐related factors. Previous studies have demonstrated poor levels of agreement between pediatric oncology patients and their caregivers on their responses to Patient Reported Outcome Measurement System (PROMIS) assessments of mobility, pain interference, psychological distress, depression, anxiety and fatigue [7] as well as questions regarding the presence of cancer‐related symptoms and beliefs about the value of palliative care [8]. Other studies have investigated pediatric patients in acute pain and found that parents tend to underestimate the severity of their children's pain [9] and variable levels of agreement between these groups when assessing satisfaction with post‐operative pain management with the weakest correlation being for ratings of worst pain [10]. Given the discrepancies between pediatric patient perceptions and those of their caregivers, the importance of obtaining both perspectives whenever possible is important to help inform the clinical decision‐making process. In situations where this is not possible, such as when the patient is too young or ill to respond, the use of a caregiver‐specific assessment tool that has been validated based on patient–caregiver agreement has been suggested [11].

While patient‐reported impressions of change have been validated to assess to what extent a treatment has been helpful in one or more domains among pediatric populations with non‐malignant chronic pain [12], there has been little research to understand how patient/caregiver perspectives compare in a pediatric population with chronic pain. Birnie et al. [13] examined the relationships between children's responses on pediatric PROMIS measures of anxiety, depression, fatigue, pain interference, mobility, and peer relations and their parent or caregiver's responses on the corresponding parent proxy PROMIS versions of these measures assessed prior to treatment in an interdisciplinary outpatient pediatric pain program. There were moderate to good interclass correlations, but discrepancies were present in more than 20% of the dyads on all PROMIS domains, with parents/caregivers tending to endorse greater severity of symptoms. Additionally, one small study of children undergoing outpatient treatment for chronic pain identified themes endorsed by patients and their caregivers during treatment, but focused on qualitative analytics rather than determining extent of agreement between the two groups [14]. In fact, no studies to date have assessed child–caregiver/parent levels of agreement on treatment outcomes following a pediatric interdisciplinary pain management program (IPMP). Given that parents typically initiate treatment for their children and that they sometimes serve as proxies for their children, it is important to understand whether or not parents and their children evaluate treatment outcomes similarly.

The primary objective of this study was to assess how well parents' perceptions of their child's improvements aligned with their child's perceptions of improvements following an IPMP via the multidimensional impression of change (MPIC) [15].

2. Materials and Methods

2.1. Participants

A retrospective review of existing data from 172 pediatric patients with chronic pain and their parents was performed. The patients participated in the program from August 2013 to September 2018. The program is located within an academic rehabilitation hospital. This study was approved by the Northwestern University Institutional Review Board which included waivers of HIPAA authorization and consent process, and procedures were conducted according to the IRB approved protocol and HIPAA law. To ensure confidentiality, all data were de‐identified. Inclusion criteria required that patients were ≤ 18 years of age, completed the IPMP, and had program assessments completed by both the patient and their parent. In order to minimize bias and provide a representative sample of patients treated in the program, no additional inclusion or exclusion criteria were used.

2.2. Procedure

All patients participated in comprehensive IPMP.

2.3. Interdisciplinary Pain Management Program

Following an evaluation by the physician and psychologist, appropriate patients are seen in the outpatient program for treatment. Patients were deemed appropriate for the program if they were < 18 years of age, had experienced pain for 3 months or longer, and were psychiatrically stable with no suicidal intent. All patients participate in weekly 1‐h physical therapy, occupational therapy, and psychological counseling sessions, plus a 15‐min meeting with the physician. Sessions focused on teaching patients tools they could use for pain management, including exercises, sensory tools, pacing, body mechanics, relaxation techniques, and cognitive behavioral therapy. The goal is to reduce fear of pain, teach pain coping techniques, improve physical functioning, and address the emotional needs of both patients and parents. Therapists worked collaboratively to help each patient incorporate these tools effectively. Therapists also reviewed these tools with parents and encouraged parents to support their children in practicing these tools and techniques. Multiple studies have reported that the most effective treatment approaches for chronic pain incorporate the physical and psychological factors that are inherent in chronic pain [16, 17, 18]. A systematic review has highlighted the positive impact that IPMP has on the functioning of children with chronic pain [19]. Appropriate patients also participate in weekly 1‐h group treatments with each discipline. Group participation is based on age (12–18) and insurance approval. Children ages 12–18 were included in group treatment, as the groups were psychoeducational and it was felt that children of this age would be able to participate in these groups, which required some higher‐level cognitive functioning. Also, patients of this age were able to relate to each other and develop peer support. Government‐based insurance programs do not provide coverage for group treatment. Most patients are in the program for 6–8 weeks, depending on progress, with slightly longer lengths of stay for those requiring additional treatment to reach their goals. Parents are asked to attend the initial and discharge sessions during which the plan of care, treatment goals, and discharge recommendations are communicated. Parents, when available, are included at the end of each session. Parents are urged to act as “coaches” for their children, supporting their child in problem solving and using pain management tools. Parental education includes best ways to provide support, and encouraging them to focus on their child's functional behaviors and avoid asking about pain.

2.4. Measures

All patients provided the following characteristics: age, sex assigned at birth, pain duration, current pain intensity, education, primary pain location/diagnosis, and primary caregiver (parent). The patients and their parents completed the Multidimensional Patient Impression of Change following the IPMP as part of routine clinical care.

2.4.1. Multidimensional Patient Impression of Change

The MPIC [15] is based on the Patient Global Impression of Change [20]. The measure assesses perceived improvements in Overall Status, Pain, Sleep, Mood, Physical Functioning, Cope with Pain, Manage Pain Flare‐ups, and Medication Effectiveness. The eight domains are rated on a 7‐point scale ranging from Very Much Improved = 1 to Very Much Worse = 7, with No Change = 3. The MPIC has been validated with both adult and pediatric patients with chronic pain [12, 15]. Both versions have good reliability with internal consistencies assessed via Cronbach's alpha yielding alpha coefficients of 0.85 and 0.89 [12, 15].

The MPIC was slightly altered in the parent version (e.g., my child's overall pain vs. my overall pain).

2.5. Statistical Analyses

Data were analyzed using IBM SPSS Statistics (Version 26, Chicago, IL). Descriptive statistics, frequencies and percentages for categorical data and means and standard deviations for continuous data were used to assess patient characteristics and MPIC responses. Mean differences among the responses on the MPIC were analyzed via paired‐samples t‐tests. Since the parent version of the MPIC had not been previously validated, the reliability of both versions was assessed using a measure of internal consistency (Cronbach's alpha) and principal components analyses. The levels of agreement on the MPIC were analyzed between child and parent using free‐marginal kappa statistic and the overall percent agreement. Randolph's free‐marginal kappa statistic was used, as the raters were not limited to assigning their ratings to a fixed number of positive responses [21, 22]. Both the free‐marginal kappa values and overall percent agreement were calculated using the online kappa calculator [23]. Kappa values range from −1.0 to +1.0, representing perfect disagreement and perfect agreement, respectively. A value of 0.0 represents an equal to chance level of agreement. Kappa values can be further classified as Poor (< 0.00), Slight (0.00–0.20), Fair (0.21–0.40), Moderate (0.41–0.60), Substantial (0.61–0.80), and Almost Perfect (0.81–1.00) [24]. The statistical significance level was set at p < 0.05 for all analyses.

3. Results

There was complete data for all but one MPIC domain (Medication Effectiveness) which was omitted by 9 (5.23%) patients and 26 (15.11%) parents, resulting in data for 145 pairs (84.3%). Missing data was likely due to the patient not taking any medications for their pain. Prior to conducting statistical analyses, the data were examined for normality. All MPIC variables on both the patient and parent versions were not normally distributed (all Shapiro–Wilk < 0.001). Analyzing the data with the nonparametric alternative to the paired‐samples t‐test (Wilcoxon Signed Rank Test) yielded no difference in significance or interpretation; thus, all analyses were conducted using dependent paired‐samples t‐tests.

3.1. Patient Characteristics

The majority of the patients were female (77.3%), and their ages ranged from 8 to 18. Their education ranged from 3rd grade to freshman in college. They reported an average pain duration of 2.42 years. On an 11‐point verbal pain rating scale (0–10), patients endorsed a moderate amount of baseline pain. The most frequent diagnostic category was headache (18.6%). Patient characteristics are presented in Table 1.

TABLE 1.

Patient characteristics.

Characteristics	Participants (N = 172)
Age (M ± SD)	13.90 (2.47)
Baseline pain (0–10)	6.55 (1.94)
Pain duration, years (M ± SD)	2.42 (2.92)
Education, years (M ± SD)	8.55 (2.51)
Sex (% female)	77.3%
Primary diagnoses, %
Headache	18.61%
Complex regional pain syndrome	13.37%
Other chronic pain (central sensitization)	12.21%
Abdominal pain	9.88%
Joint pain	8.72%
Back pain	7.56
Myofascial pain	5.80%
Hypermobility	4.65%
Fibromyalgia	4.65%
Functional Neurological Disorder	4.65%
Ehlers–Danlos syndrome (EDS)	3.49%
Neuropathic pain	3.49%
Scoliosis	1.16%
Other	1.74%

3.2. Reliability Analyses

Principal components analyses were used to assess the factor structure of both versions of the MPIC. An eigenvalue of 1.0 was used for factor extraction. One‐factor solutions were found for both versions of the MPIC, accounting for 50.9% of the variance in the patient version and 55.7% of the variance in the parent version. Factor loadings ranged from 0.61 to 0.79 for the patient version, and 0.56 to 0.85 for the parent version. All factor loadings are presented in Table 2. Cronbach's alpha showed good internal consistency on both the patient (α = 0.85) and parent (α = 0.87) versions of the MPIC.

TABLE 2.

Principle components analysis factor loadings for the MPIC.

Domain	Patient	Parent
1. Overall status	0.78	0.85
2. Pain	0.71	0.70
3. Sleep	0.61	0.67
4. Mood	0.72	0.77
5. Physical functioning	0.70	0.78
6. Cope w/pain	0.78	0.83
7. Manage Pain Flare‐ups	0.79	0.78
8. Medication effectiveness	0.61	0.56

3.3. MPIC Responses

More than half (≥ 57%) of the patients perceived themselves to be much or very much improved on all domains except for overall Pain (49.4%) and Medication Effectiveness (49%). Similarly, more than half (≥ 55.8%) of the parents perceived their child as much or very much improved in all domains except for Medication Effectiveness (47.6%). Overall Status, Cope with Pain, and Physical Function were the three domains most frequently rated much or very much improved (patient 79.7%–80.8%, parent 79.1%–82.5%). Contrariwise, very small percentages of both patients (0%–4.7%) and their parents (0%–3.5%) rated the patients as minimally worse to very much worse on all eight domains, with the dyads reporting no worsening in Physical Functioning or Cope with Pain. Additional details are provided in Figure 1.

FIGURE 1.

MPIC response rates.

3.4. Overall Patient–Parent Agreement

The overall percent agreements and free‐marginal kappa's (K _free) are described in Table 3. The levels of agreement for all eight domains were statistically significant (p's < 0.001). Kappa values indicated Fair levels of agreement for Manage Pain Flare‐ups and Mood (K _free = 0.36, 0.38), and Moderate levels of agreement for the remaining six domains (K _free = 0.42–0.53).

TABLE 3.

Levels of agreement by MPIC domain.

Domains	Parent/child
	K free [95% CI]	PA
Overall status	0.46* [0.37, 0.54]	53.5%
Pain	0.53* [0.44, 0.61]	59.3%
Sleep	0.42* [0.33, 0.50]	50.0%
Mood	0.38* [0.30, 0.47]	47.1%
Physical function	0.47* [0.38, 0.56]	54.7%
Cope w/pain	0.46* [0.38, 0.55]	54.1%
Manage Pain Flare‐up	0.36* [0.28, 0.45]	45.4%
Medication effectiveness	0.44* [0.34, 0.53]	51.7%

Abbreviation: PA, total percent agreement.

^* p < 0.0001.

3.5. MPIC Rating Comparisons

The differences in mean MPIC ratings were statistically different for Pain, Mood, and Cope with Pain. Parents endorsed higher levels of improvement than did their children for Pain (t = 2.470, p = 0.014) and Mood (t = 3.563, p < 0.001). Conversely, the patients rated themselves with higher levels of improvement than did their parents in ability to Cope with Pain (t = −2.296, p = 0.023). Descriptive data are presented in Table 4.

TABLE 4.

Patient and parent average MPIC ratings.

MPIC domain	Patient rating mean (SE)	Parent rating mean (SE)	Mean difference [95% CI]
Overall status	1.87 (0.07)	1.85 (0.06)	0.017 [−0.11, 0.14]
Pain	2.65 (0.10)	2.49 (0.09)	0.157* [0.03, 0.28]
Sleep	2.44 (0.10)	2.38 (0.09)	0.064 [−0.09, 0.22]
Mood	2.44 (0.09)	2.17 (0.08)	0.267** [0.12, 0.42]
Physical function	1.85 (0.07)	1.89 (0.06)	−0.041 [−0.16, 0.08]
Cope w/pain	1.77 (0.06)	1.91 (0.07)	−0.140* [−0.26, −0.02]
Manage Pain Flare‐up	2.01 (0.07)	2.02 (0.06)	−0.012 [−0.14, 0.12]
Medication effectiveness a	2.68 (0.10)	2.68 (0.10)	−0.007 [−0.19, 0.18]

^a n = 145.

^* p < 0.05.

^** p < 0.01.

4. Discussion

Overall, these results showed that both the patient's and their parents endorsed improvement on all domains of the MPIC following the interdisciplinary pain management program. There was statistically significant agreement between patients and parents for all domains. Moreover, Mood and ability to Manage Pain Flare‐ups had fair agreement, while Overall status, Pain, Sleep, Physical Function, Cope with Pain, and Medication Effectiveness had moderate levels of agreement. Comparatively, parents endorsed higher levels of improvement in Pain and Mood than did their children, and children rated greater improvement in their ability to Cope with Pain than their parents.

Self‐reported outcomes from the actual patient whenever possible are the most important information to have, but among pediatric patients, caregiver reports are of unique value [25] and are an important factor in decision‐making and implementation of treatment plans [26]. The level of agreement between pediatric patients with chronic pain and their parents specifically on the MPIC has not previously been assessed but has been investigated prior to enrollment in an interdisciplinary program using PROMIS pediatric self‐report and parent proxy measures of anxiety, depression, fatigue, pain interference, and mobility [13]. The results, based on interclass reliability, demonstrated a moderate level of agreement for pain interference and good reliability for all other domains [13]. Moreover, examination of the individual items on these instruments administered to a general pediatric population showed lower levels of agreement for items related to relationships, anger, anxiety, and depressive symptoms with better agreement on items related to energy, fatigue, pain interference, and mobility [27]. Further analysis of this data demonstrated that when parents or children had chronic pain, there was better agreement in pain interference items [27]. Direct comparisons to this existing data are difficult because of differences in statistical methods; however, taken in total, these studies all demonstrate non‐perfect levels of agreement supporting the idea that unique information is obtained from both children and caregivers and that this information is not interchangeable [27, 28, 29, 30].

In our study, there tended to be weaker agreement (K _free = 0.38) when assessing mood symptoms (internal, less visible variable) and stronger agreement (K _free = 0.47) when assessing Physical Function (external, more visible variable) which is consistent with the results from studies of patients in chronic pain [13] and the general population [27], however, this may not be the case with other medical diagnoses. Some studies have shown higher parent–caregiver agreement for psychosocial functioning among patients with cancer [31], rheumatological disorders [32], and congenital heart disease [33] while others have shown stronger agreement for physical functioning in these populations [7, 34, 35]. Upton et al. [36] have postulated that the level of agreement may not depend so much on the domain itself and how outwardly visible it is, but instead the clinical relevance of a given domain to a given disease group, with higher levels of agreement being present for domains that are more important for a given diagnosis. The results of this study did not fully support that theory. Mood disturbances (particularly depression) co‐occur at much higher rates in children with chronic pain than in children without chronic pain, but the level of agreement for Mood was one of the lowest found in our study (K _free = 0.38) [37, 38, 39, 40]. The ability to manage pain flare‐ups is integral to successful long‐term symptom management and was found in this study and one other [12] to be at least “Much improved” after treatment; yet it was not represented in the group of domains with the highest level of patient–parent agreement (K _free = 0.36).

In addition to slight differences in the level of agreement, there were also some key differences in the magnitude of perceived improvement between parents and their children. Parents perceived greater improvement in Pain and Mood, while children endorsed higher improvement in ability to Cope with Pain, which is in agreement with some, but not all prior studies. Previous studies investigating similar relationships focused on reports of symptoms such as severity of pain, depression, anxiety, physical function, disability, and quality of life. For example, some studies have shown more frequent caregiver overestimation of symptom severity compared to their children's self‐report, both in physical and emotional components of pain, as well as an underestimation of their health‐related quality of life [13, 25, 30, 41]. However, in children with headaches, the frequency and severity of symptoms were consistently reported as worse by children as compared to their parents [42]. The over‐ and underestimates of the children's symptomatology in these studies were assessed at a single point in time, and none of these studies assessed perceived improvement following treatment. It is important to be aware of these differences because overestimation or underestimation of children's pain and function may lead to unnecessary treatment or inappropriate early discharge from necessary care. Direct comparison to the results of this study is difficult given the introduction of the element of time when assessing improvement; however, if the symptoms are perceived as more severe at baseline, one could argue there is a potential for a larger magnitude of improvement, which may account for our observations of greater parental perceptions of improvement in Pain and Mood.

Pain and Mood are both generally well‐understood concepts that would not require special explanation before expecting a fair response with regard to perceived improvement by parents or patients. However, the ability to Cope with Pain is a domain that children are more likely to be more familiar with than their parents, as it is emphasized as a goal of care throughout the course of their treatment. The fact that patients tended to report a greater magnitude in improvement in this domain may simply be a function of their level of familiarity with it combined with the expectation that this should improve over the course of care—a concept that may not necessarily be emphasized with the parents. This would be consistent with previous parental reports of difficulty responding to some parent proxy scales due to limited information being shared with them by the patient [43]. Some previous studies have specifically looked at parent and patient factors that may be influencing these discrepancies. For example, it has been shown that greater pain catastrophizing, whether experienced by the caregiver or child, was significantly associated with greater symptom reporting [13]. Additionally, caregivers with worse physical health reported more concerns with child mobility compared to child self‐report and caregivers with poorer mental health reported more concerns with child anxiety, depression, and peer relations as compared with child self‐report [13]. Furthermore, mothers with higher levels of distress have been shown to report more child symptoms than their children do [41]. Taken together, this evidence supports the notion that parents' own medical and psychological status influences their beliefs and perceptions of their children's symptoms. However, when assessing quality of life rather than symptom severity or health status, there is some contradictory evidence suggesting that parent quality of life does not affect their reporting of quality of life in their children with chronic pain [44]. Some factors including age, gender, race, symptom duration and severity, household income, and parental marital status have not been found to consistently influence patient–parent agreement [42, 44]. While the current study was not specifically designed to identify these potentially influential parent and patient factors, it will be of interest to investigate this in future studies.

There were several important limitations in this study. First, all data was collected from a single tertiary care specialty academic center to which patients are typically referred only after they have failed to improve with community‐based treatment, which may limit the generalizability of the results. Second, as discussed, there is likely some bias introduced to the patients' perspective (that may affect parents depending on their level of involvement in care and communication with their child) given the educational nature of the treatment program with an emphasis on self‐management and acceptance of symptoms that likely influences how they respond to the MPIC. For example, the stated goal of the program isn't necessarily to reduce the severity of pain, but rather to improve their ability to cope with symptoms and remain as functional as possible. With an understanding of these goals, the patients are likely biased to respond accordingly, whereas parents less familiar with these concepts may simply interpret an overall improvement as being secondary to reduced pain and report that. Third, this study was not designed to identify parent or patient characteristics or beliefs that have been shown in other studies to influence responses. This would be an important future investigation to consider. Lastly, due to this being a real‐world observational study, there was no control group to measure the outcomes against. However, this study was not designed to evaluate program efficacy, but rather to assess if the patients and their parents viewed their outcomes similarly.

Overall, the MPIC was found to be an effective means of assessing perceived improvements in the pediatric chronic pain population with fair to moderate agreement between children and parents on all domains. Parents endorsed higher levels of improvement in Pain and Mood than did their children and children rated greater improvement in their ability to Cope with Pain than the parents, which itself reflects effective communication of program goals to the patients. The differences on some of these domains suggest the need for ongoing parent–child dyad assessments. These discrepancies can function as the basis for end of program discussions among providers, patients and parents. Furthermore, parents may benefit from more in‐depth education regarding the focus of interdisciplinary pain management programs (e.g., rehabilitation rather than cure). Future studies are needed to better understand what parent and patient factors are driving the differences that were observed. Thereafter the utility of incorporating both perspectives in the development of treatment plans and assessment of their success can better be determined.

Funding

The authors have nothing to report.

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

Research data are not shared.