Congruence of patient- and clinician-reported toxicity in women receiving chemotherapy for early breast cancer

Abstract

Background.

The National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE), collected alongside clinician-reported CTCAE, enables comparison of patient and clinician reports on treatment toxicity.

Methods.

In a multi-site study of women receiving chemotherapy for early-stage breast cancer, symptom reports were collected on the same day from patients and their clinicians for 17 symptoms; their data were not shared with each other. Proportions of moderate, severe or very severe patient-reported symptom severity were compared with clinician-rated toxicity grades 2, 3 or 4. Patient-clinician agreement was assessed via Kappa statistics. Chi-square tests investigated whether patient characteristics were associated with patient-clinician agreement.

Results.

Among 267 women, median age was 58 (range 24–83) and 26% were non-white. There was moderate scoring agreement (Kappa 0.413 – 0.570) for 53% of symptoms, fair agreement for 41% (Kappa range 0.220 – 0.378), and slight agreement for 6% (Kappa 0.188). For example, for fatigue, patient-reported and clinician-rated percentages were 22% vs 8% severe or very severe, 41% vs 46% moderate, 32% vs 39% mild, and 6% vs 7% none. Clinician severity scores were lower for non-white compared to white patients for peripheral neuropathy, nausea, arthralgia and dyspnea.

Conclusion.

Although clinician reporting of symptoms is common practice in oncology, there is suboptimal agreement with the gold standard of patient self-reporting. These data provide further evidence supporting the integration of patient-reported outcomes into oncological clinical research and clinical practice to improve monitoring of symptoms as well as timely intervention on symptoms.

Clinical Trial Registration:

NCT02167932, NCT02328313, NCT03761706

INTRODUCTION

The National Cancer Institute’s (NCI) Common Terminology Criteria for Adverse Events (CTCAE)¹ is the long-standing standard approach for collection and reporting of adverse events (AEs) in oncology research². Of the approximately 800 AEs included in the CTCAE item library, approximately 10% correspond to symptoms, such as nausea and sensory neuropathy. However, CTCAE items are recorded by clinical research staff rather than by patients. In response to growing evidence of the value of patient-reported symptom severity as a complement to clinician-assessed toxicity³, the NCI supported the development of a patient-reported outcome (PRO) version of the CTCAE, called the PRO-CTCAE^4,5, which became publicly available in April 2016. Like CTCAE, PRO-CTCAE provides single-item measures for patient-reported symptom “severity”, but also includes items for “interference with usual or daily activities” and “frequency” of some symptoms.

The development of PRO-CTCAE held the promise of improved understanding of patient and clinician toxicity reports for multiple symptoms simultaneously and at multiple time points during chemotherapy, if PRO-CTCAE and CTCAE reports were completed in real time during the same clinic visit⁶. This would enable more rigorous analyses of convergence and divergence in patient and clinician perspectives on important clinical endpoints such as quality of life and function⁷. The PRO-CTCAE could also facilitate collaborative reporting on symptoms that are not asked about routinely, through a process in which patient-reported toxicity forms are made readily available to the treating clinician during routine clinic visits⁸.

Within a growing body of literature documenting discrepancies between patient- and clinician-reported toxicities⁸, studies making paired comparisons of health care provider-assessed CTCAE and patient-reported PRO-CTCAE (or patient-tested precursors to the PRO-CTCAE) have been conducted in patients receiving chemotherapy for head and neck cancer⁹, genitourinary cancer¹⁰, and lung cancer^10,11, patients receiving radiotherapy¹², and patients receiving chemotherapy and/or radiation therapy⁵, with four of these studies collecting data at more than one time point during treatment^5,9,11,12. Additional studies have compared CTCAE toxicity grades with validated symptom measures such as the European Organization of Research and Treatment of Cancer Core Quality of Life Questionnaire (EORTC QLQ-C30)^13–15 or other study-specific symptom reports¹⁶, with one of these studies collecting data at multiple time points during treatment¹⁴. Some of the afore-mentioned studies included women with early breast cancer within a mixed sample of adults with cancer; however, findings were not reported separately for each type of cancer^5,13,16 or had a specific focus on early-stage breast cancer.

In this study, we conducted an analysis among women with early-stage breast cancer in which we compared clinician-reported (CTCAE) and patient-reported (PRO-CTCAE) severity for 17 symptoms collected at multiple time points throughout chemotherapy. We have previously reported that patient-assessed symptom severity for these 17 symptoms varies significantly among four chemotherapy regimens commonly used in current clinical practice¹⁷, confirming the importance of continuous symptom monitoring throughout treatment. We have also reported that there is minimal agreement between patient- and clinician-reported severity scores for chemotherapy-induced peripheral neuropathy associated with these chemotherapy regimens¹⁸. In the current study, we compare patient and clinician reports for all 17 symptoms, and we identify factors that may be associated with patient-clinician consensus or divergence¹³.

PATIENTS AND METHODS

Study participants

This is a secondary analysis of data on a sample of women recruited into one of three prospective non-randomized studies of a walking intervention for patients receiving (neo)adjuvant chemotherapy for early breast cancer (Stage 0–3) (American Joint Committee on Cancer staging, 7th edition) (ClinicalTrials.gov identifiers NCT02167932, NCT02328313, and NCT03761706). Patients were ages 21 years or older and recruited prior to starting chemotherapy regimens that were selected by clinicians in consultation with their patients. Patients provided written informed consent, and the studies were approved by the University of North Carolina Lineberger Comprehensive Cancer Center Protocol Review Committee and the Institutional Review Boards for each study site.

Measures

From chemotherapy initiation through end of chemotherapy, patients completed a patient-reported symptom form for 17 symptoms. These symptoms were selected a priori for their observed frequency in the treatment of patients with early breast cancer. In two studies (NCT02167932, NCT02328313), the reporting form was the validated patient-reported symptom monitor (PRSM)¹⁹. The PRSM was a pre-cursor to the PRO-CTCAE, and was used because PRO-CTCAE was not publicly available when these two studies were initiated. The PRSM pre-cursor was developed by investigators who were also involved with the development of PRO-CTCAE^4,20 and has an analogous structure and response scale to the PRO-CTCAE (Appendix 1), employing single-item measures of symptom severity on a 5-point scale with response options from none/no symptom to very severe²¹. In addition, patients reported symptom “interference with doing things you usually do” using a single-item measure, with similar 5-point response options from not at all to very much. When PRO-CTCAE became publicly available, it was used as the reporting form for the third study (NCT03761706). Depending on their chemotherapy infusion schedule over 4 to 8 total cycles, patients completed symptom reports every other week or every third week. Patients with weekly infusion schedules (mostly paclitaxel) completed symptom reports every other week, to avoid over-reporting in this cohort compared to the rest of the sample. Patients completed symptom reports during their chemotherapy infusion, which was after they had seen their oncology clinician.

On the same day that patients completed symptom reports, their oncology clinician (MD, Nurse Practitioner, or Physician Assistant) was asked to complete a CTCAE study form to rate the same set of 17 symptoms. The patient reports were not available to their clinicians. For comparison with patient-reported scores, CTCAE response options were standardized across symptoms as follows: 0=none, 1=mild, 2=moderate, 3=severe, and 4=disabling^22,23. We matched patient-reported “none” with CTCAE grade 0, “mild” with grade 1, “moderate” with grade 2, and “severe/very severe” with grade 3/4^21,24, consistent with a previously developed mapping algorithm^22,23.

Statistical analysis

Descriptive statistics were used to summarize patient characteristics, breast cancer diagnosis and treatment, adverse events, and patient- and clinician-reported symptom scores. Because clinicians were not always available to complete reports, only data points from days where both the patient and the clinician reported are included. The essential metric for our study was the maximum score for each symptom at any time during the measurement period (start to end of chemotherapy), which is the approach used in clinical trials when reporting treatment toxicity. Proportions of moderate/severe/very severe patient-reported symptom “severity” and “interference” were compared with clinician-rated toxicity grades 2/3/4 for all 17 symptoms combined at each level of severity (none, mild, moderate, severe/very severe) for each symptom individually.

Agreement between patient- and clinician-reported dichotomized maximum scores was assessed by reporting simple Kappa coefficients for each symptom²⁵. Dichotomization was “low” for none or mild and “high” for moderate, severe or very severe. A priori interpretation of the Kappa statistic used standard rating criteria²⁶: < 0.0 less than chance agreement, 0.01 – 0.20 slight agreement, 0.21 – 0.40 fair agreement, 0.41 – 0.60 moderate agreement, 0.61 – 0.80 substantial agreement, and 0.81 – 0.99 almost perfect agreement. The same method was used to compare patient-reported symptom “interference with things you usually like to do” with clinician toxicity grade. Chi-square tests were conducted to investigate whether patient characteristics were associated with patient-clinician agreement on maximum severity scores for each symptom individually.

RESULTS

Patient Characteristics

In a sample of 267 women, median age was 58 years (range 24 – 83), of whom 26% were non-white. Breast cancer was distributed across stages I, II and III and across four different common chemotherapy regimens (Table 1). A total of 1203 same-day paired reports were considered in our analysis, and the maximum symptom score for each patient was the unit of analysis. For patients receiving AC-T (doxorubicin/cyclophosphamide followed or preceded by paclitaxel/Taxol) or AC-TC (doxorubicin/cyclophosphamide plus paclitaxel/carboplatin), the median number of reports was 6, for TC (docetaxel/cyclophosphamide plus/minus anti-HER2 therapy) it was 3, for TCH (docetaxel/carboplatin plus anti-HER2 therapy) it was 5, and for all regimens it was 4.

Table 1.

Study Participant Characteristics (N=267)

Variable
Age	58 (SD 13) Range 24 – 83
Race
Not white	70 (26%)
White	197 (74%)
Education
High school or less	38 (14%)
More than high school	227 (86%)
Married
No	116 (44%)
Yes	149 (56%)
Body Mass Index (BMI) – mean SD	30 (SD 7) Range 17 – 65
Underweight (<18.5)	3 (1%)
Normal (18.5 to <25)	72 (27%)
Overweight (25 to <30)	83 (31%)
Obese I (30 or above)	109 (41%)
Menopausal status at breast cancer diagnosis
Pre-menopausal	81 (31%)
Post-menopausal	183 (69%)
Breast cancer stage
I	67 (25%)
II	133 (50%)
III	67 (25%)
Breast cancer phenotype
HR negative/HER-2 negative	78 (29%)
HR negative/HER-2 positive	34 (13%)
HR positive/HER-2 negative	120 (45%)
HR positive/HER-2 positive	34 (13%)
Breast cancer surgery
None	7 (3%)
Lumpectomy	126 (48%)
Mastectomy	127 (49%)
Anti-HER-2 therapy	67 (25%)
Chemotherapy timing
Neoadjuvant	103 (39%)
Adjuvant	159 (60%)
Both	1 (1%)
Chemotherapy regimens -- drug combinations
AC-T (doxorubicin/cyclophosphamide followed or preceded by paclitaxel/Taxol)	82 (31%)
AC-TC (doxorubicin/cyclophosphamide plus paclitaxel/carboplatin)	19 (7%)
TC (docetaxel/cyclophosphamide; N=3 plus anti-HER-2 therapy)	70 (27%)
TCH (docetaxel/carboplatin plus anti-HER-2 therapy)	41 (16%)
Other	51 (19%)

HR=hormone receptor. HER-2: Human Epithelial Growth Factor receptor 2.

Patient and clinician symptom severity scores

In Figure 1, proportions of patient-reported maximum severity scores ranging from none to severe/very severe and clinician toxicity grades from 0 to 3/4 are presented for individual symptoms. For example for fatigue, patient- compared to clinician-rated percentages were 22% vs 8% severe/very severe, 41% vs 46% moderate, 32% vs 39% mild, and 6% vs 7% none. This figure illustrates how proportions of patient-reported moderate and severe/very severe symptoms are consistently higher than clinician-rated toxicity grades 2 and 3/4.

Figure 1.

Maximum severity score at any time during chemotherapy – patient and clinician scores (percent)

Figure 2 illustrates the percent of patients who rated their symptom severity or interference as moderate, severe or very severe and clinicians rated their toxicity grade 2, 3 or 4. For example, in the far right grouping, 46% of patients rated more than four symptoms as moderate or worse severity; 34% of patients rated more than four symptoms as moderate or worse interference; and 27% of patients were rated by their clinician as having more than four symptoms graded 2, 3 or 4. In the far left group, 15% of patients rated none of their symptoms as moderate or worse severity, 27% of patients rated symptom interference as moderate or worse, and 19% of patients had none of their symptoms graded by their clinicians as 2 or higher.

Figure 2.

Patient-reported moderate/severe/very severe symptom severity and interference and clinician toxicity grade 2/3/4 (percent of patients)

Agreement of patient and clinician rated symptom severity

Table 2 shows the proportions of study participants where patients and clinicians agreed that symptom severity was “low” (none, mild) or “high” (moderate, severe/very severe). The table also shows where clinician maximum severity scores were higher than patient scores (clinician high/patient low), and where patient maximum scores were higher than clinician scores (patient high/clinician low). For example, for constipation, there was patient-physician agreement on “low” symptom severity for 65% of patients and agreement on “high” symptom severity for 11% of patients; however, for 3% of patients their clinicians rated symptom severity higher than their patients and, in turn, 21% of patients rated their constipation severity higher than their clinician.

Table 2.

Agreement between patients and clinicians -- maximum symptom “severity” and ‘interference” scores at any time during treatment (percent) (N=267)

Agreement on maximum symptom “severity” score
Symptom	Agree “low”	Agree “high”	Clinician “high”, patient “low”	Patient “high”, clinician “low”	Kappa*
Constipation	65	11	3	21	.329
Diarrhea	62	14	4	21	.378
Nausea	65	14	7	14	.437
Vomiting	90	2	4	5	.220
Mucositis oral	74	10	5	12	.447
Fatigue, lack of energy	31	48	7	14	.570
Aching joints/arthralgia	60	17	5	18	.455
Aching muscles/myalgia	62	13	6	20	.363
Peripheral neuropathy	60	16	11	13	.413
Anxiety	57	16	5	22	.372
Feeling sad, unhappy/depression	73	10	2	15	.444
Insomnia	43	27	5	25	.416
Dyspnea/light-headedness	77	4	4	15	.224
Abdominal pain	83	2	2	14	.188
Edema limbs	79	4	1	17	.245
General pain	60	20	8	12	.523
Hot flashes	66	16	8	11	.513
Agreement on maximum symptom “interference” score
Symptom	Agree “low”	Agree “high”	Clinician “high, patient “low”	Patient “high”, clinician “low”	Kappa*
Constipation	77	6	7	9	.337
Diarrhea	70	11	6	13	.426
Nausea	68	11	9	12	.384
Vomiting	89	1	4	6	.150
Mucositis oral	80	6	9	6	.348
Fatigue, lack of energy	32	44	11	13	.522
Aching joints/arthralgia	64	14	9	14	.402
Aching muscles/myalgia	68	12	7	14	.406
Peripheral neuropathy	64	10	18	8	.256
Anxiety	68	13	9	11	.441
Feeling sad, unhappy/depression	79	8	4	9	.506
Insomnia	52	21	12	16	.398
Dyspnea/light-headedness	78	4	4	12	.275
Abdominal pain	85	3	1	11	.265
Edema limbs	86	2	2	10	.247
General pain	61	18	11	10	.466
Hot flashes	71	8	16	5	.311

“Low” = patient-reported none or mild; clinician rated toxicity grade 0 or 1

“High” = patient-reported moderate, severe or very severe; clinician-rated toxicity grade 2, 3 or 4

Kappa interpretation: < 0.0 less than chance agreement, 0.01 – 0.20 slight agreement, 0.21 – 0.40 fair agreement, 0.41 – 0.60 moderate agreement, 0.61 – 0.80 substantial agreement, and 0.81 – 0.99 almost perfect agreement.

^*All kappas were statistically significant (p=<.05)

Overall, there was “moderate” agreement (Kappa 0.413 – 0.570) on symptom severity for 9 of 17 (53%) symptoms, “fair” agreement on 7 symptoms (41%) (Kappa 0.220 – 0.372), and “slight” agreement on one symptom (6%) (Kappa 0.188). In the lower half of Table 2, we report the comparison of patient-reported symptom interference with clinician-reported severity. Again, we find “moderate” agreement (Kappa 0.402 – 0.522) on 7 of 17 symptoms (41%), “fair” agreement on 9 symptoms (53%) (Kappa 0.247 to 0.398), and “slight” agreement on one symptom (6%) (Kappa 0.150). All Kappa estimates were statistically significant (p<=.05)

Variables associated with patient-clinician agreement on symptom severity scores

Using three levels of agreement (agree, clinician high/patient low, and patient high/clinician low), associations with patient characteristics were explored (Appendix 2). The highest number of statistically significant associations – signifying differences between patient and clinician scores – was seen for race with regard to nausea (p=.05), arthralgia (p=.04), peripheral neuropathy (p=.04), and dyspnea (p=.05). These differences are further elucidated in Figure 3, which shows that in 21% of non-white patients as compared to 10% of white patients, clinicians rated peripheral neuropathy severity “low” when patients rated it “high” (p=.04). However, the reverse is shown for nausea where clinicians rated symptom severity “low” when patients rated it “high” in 15% of white compared to 12% of non-white patients (p=.05).

Figure 3.

Patient-clinician congruence by race and Body Mass Index (BMI) (percent)

Similarly, Figure 3 presents significant differences by BMI, with clinician severity scores for constipation lower than patient scores for patients with BMI less than 25 (14%), BMI 25–30 (35%), and BMI 30 or higher (17%) (p=.003). BMI-related differences are also shown for edema (p=.005), with the rate of clinician under-reporting increasing with increasing BMI levels. Regarding marital status, clinician severity scores for peripheral neuropathy were lower for unmarried (19%) compared to married (8%) patients (p=.03). There were no significant differences for age, education or menopausal status.

DISCUSSION

Quality of life has been measured extensively in women with early breast cancer²⁷, but few studies have administered single-item symptom assessments related to specific treatment- or disease-related adverse events at frequent intervals during active treatment (which is an emerging standard for adverse event monitoring in clinical trials)⁸, or have compared same-day patient and clinician reporting of this information. In our sample of women with early breast cancer, toxicity scores for 17 symptoms were collected longitudinally using single-item scales for patient-reported symptom severity and interference (PRO-CTCAE or PRSM) and clinician toxicity grades (CTCAE). Patients completed their form before seeing their oncologist, and clinicians completed their form after the visit. Scoring reports were not shared between patients and clinicians. Analysis was limited to patient-clinician scores that were collected on the same day (“paired”).

Across all 17 symptoms, clinician toxicity grades were lower than patient-reported severity scores, as seen in the proportion of symptoms where patients rated symptom severity high while clinicians rated toxicity low. This observation corroborates findings from an Italian study in women with early breast cancer that compared symptom questionnaires from patients at two time points (using a translation of CTCAE into Italian) to toxicity grades that were extracted and interpreted from clinician notes by research staff nurses²⁸. In our study, we note higher congruence between patients and clinicians when symptoms severity was “low” and lower congruence when symptom severity was “high”, which is similar to what has been previously reported in other studies²⁹. This observation is especially problematic when patients report “high” symptom severity while their clinician notes “low” toxicity, as observed with regard to insomnia in 25% of patients, anxiety in 22% of patients, constipation in 25% of patients, diarrhea in 21% of patients, and myalgia in 20% of patients. It was exceptional when clinicians rated symptom toxicity “high” when their patients rated it “low, as observed for peripheral neuropathy in 11% of patients, general pain and hot flashes in 8% of patients, and fatigue in 7% of patients.

We investigated patient-reported scores for “interference with what you usually like to do” and found them to be substantially lower than patient-reported symptom severity. We also compared patient-reported interference with clinician toxicity scores to see if this comparison yielded greater congruence, which it did not. In our final analysis of the data, we found that patient characteristics were by and large not associated with patient-clinician disagreement on severity scores. However, we did find that clinician under-estimation of certain symptoms was greater in non-white patients as compared to white patients. This finding warrants further research, but also reflects the larger literature documenting racial disparities in patient-provider communication^30–33 as well as racial differences in symptom management experiences³⁴.

We note that patients completed their form prior to seeing their oncologist. Clinicians completed their form after the visit; however, not always immediately after seeing the patient. It is possible that substantial time lag (which we did not measure) between seeing the patient and completing the form may have affected the clinician’s recall of the patient’s symptom severity. We also did not gather data on whether the clinician form was completed by an MD, NP or PA, and therefore did not analyze potential differences among clinicians.

Patient-centered care, which is crucial to high quality health care³⁵, requires inclusion of the patient’s assessment of treatment toxicity. It is important to understand when and how patient and clinician perspectives diverge, and for which symptoms and patient characteristics. Our study points to the potential for racial disparities in symptom assessment by clinicians. The moderate or lower Kappa agreement across all 17 symptoms suggests challenges in effective patient-clinician communication about symptom experience across domains of symptom clusters (e.g., psycho-neurological, gastrointestinal, hormonal)³⁶. Disagreement in scores tends to be at the high symptom severity end of the spectrum, with clinicians underestimating severity. Continuous symptom monitoring from both patients and clinicians, from pre-chemotherapy (to establish the patient’s baseline)⁷ through end-of-chemotherapy, provides an opportunity for early intervention on symptoms for which there are pharmaceutical remedies (e.g., anxiety, depression, insomnia) or non-pharmacological remedies (such as moderate exercise to mitigate fatigue^37,38).

There is growing evidence that patients are willing and able to complete PRO-CTCAE items during a treatment-related clinic visit and after treatment has been completed³⁹. Using nurses and nurse navigators, clinics could consider developing processes to record and review patient reported symptoms and consult with the oncologist for “real time” interventions to reduce symptom severity. These processes would likely improve the likelihood of treatment completion, potentially improve patient quality of life during chemotherapy to the extent toxicities are effectively managed, and enhance overall satisfaction with care^11,40. Alternative payment models for oncology could facilitate the incorporation of patient-reported symptom assessment in quality metrics by providing reimbursement for these added responsibilities.

CONCLUSION

Although clinician reporting of symptoms is common practice in oncology, there is suboptimal agreement with the gold standard of patient self-reporting, particularly for non-white patients. These data provide further evidence supporting the integration of patient-reported outcomes into cancer research and clinical practice to improve symptom monitoring and guide timely interventions. This, in turn, would enable the timely identification of symptoms for which there are evidence-based interventions. Our findings support attention to patient-clinician interactions that are patient-centered and focus on quality of life as well as effective symptom with particular attention to cultural sensitivity^32,41. Further research is needed to explore approaches to encouraging and enabling patient-provider communication on symptom severity in ways that are actionable in “real world” clinical practice.