Assessing Patients’ Experiences with Communication Across the Cancer Care Continuum

Abstract

Objective

To evaluate the relevance, performance and potential usefulness of the Patient Assessment of cancer Communication Experiences (PACE) items.

Methods

Items focusing on specific communication goals related to exchanging information, fostering healing relationships, responding to emotions, making decisions, enabling self-management, and managing uncertainty were tested via a retrospective, cross-sectional survey of adults who had been diagnosed with cancer. Analyses examined response frequencies, inter-item correlations, and coefficient alpha.

Results

A total of 366 adults were included in the analyses. Relatively few selected “Does Not Apply”, suggesting that items tap relevant communication experiences. Ratings of whether specific communication goals were achieved were strongly correlated with overall ratings of communication, suggesting item content reflects important aspects of communication. Coefficient alpha was ≥.90 for each item set, indicating excellent reliability. Variations in the percentage of respondents selecting the most positive response across items suggest results can identify strengths and weaknesses.

Conclusion

The PACE items tap relevant, important aspects of communication during cancer care, and may be useful to cancer care teams desiring detailed feedback.

1. Introduction

“The most important goal of a high-quality cancer care delivery system is meeting the needs of patients and their families” [1]

Patient-centered communication (PCC) is key to meeting patients’ and families’ needs, and is a fundamental requirement for high quality cancer care [1]. There is a substantial and convincing body of evidence that PCC contributes to better patient outcomes [2]. However, there is also considerable evidence that PCC is not occurring consistently [3]; that poor communication is distressing to patients and damaging to the clinician-patient relationship [4], and that communication problems can contribute to physical harm and poor medical outcomes [5, 6].

The first step in improving communication during cancer care is to gather information on the current state of practice, including what is being done well, and where current practice is falling short [7–9]. Both types of information can help to drive care improvements, especially when they specify important, achievable outcomes to target. Once interventions and resources are directed to improving communication, assessments are needed to evaluate intervention effectiveness.

Interviews with patients, family members, clinicians and health care organization leaders document that stakeholders would value a communication monitoring system that would systematically assess patients’ communication experiences over time, across different phases of cancer care [10]. The recent explication of a six function model of PCC in cancer care by Epstein and Street provides a strong a conceptual framework for developing communication measures as a foundation for such monitoring systems [2].

Here we describe initial testing of a new communication measure the Patient Assessment of cancer Communication Experiences (PACE) intended to assess the patient’s perspective on communication over the course of cancer care. We evaluate the relevance, performance and potential usefulness of the PACE items, using responses collected via a retrospective, cross-sectional survey of adults who had been diagnosed with cancer. We also provide an illustrative example of how implementing these items could result in important feedback for care teams.

2. Methods

2.1 Item Development

We developed multiple sets of items to assess achievement of key communication goals across each phase of cancer care from the first suspicion of cancer through the conclusion of treatment. We did not develop items focused specifically on palliative care, hospice or end of life issues. Items were written to assess each of the six functions of the Epstein-Street PCC framework (fostering healing relationships, exchanging information, responding to emotions, making decisions, managing uncertainty, enabling patient self-management) [2]. Items focused on the patient’s perception of whether specific communication goals had been achieved, rather than whether particular communication behaviors had occurred. For example, an item such as “The doctor discussed the risks and benefits of my treatment options” focuses on behavior, rather than the goal of that behavior, patient understanding. More appropriate wording, from a communication goal perspective, would be “I understood the risks of my different treatment choices.”

We were informed by patients’ reports of their experiences with communication during cancer diagnosis and treatment in previous studies [5, 11], and their responses to early drafts of some items [10]. We also reviewed a number of existing items sets (e.g., the Consumer Assessment of Healthcare Providers and Systems [CAHPS] family of surveys [12–15] including a newly developed cancer specific CAHPS survey [16], previously drafted unpublished items developed by two of the authors (KM, RS), and prior work on PCC item development funded by the National Cancer Institute [17, 18].

Items and response options are provided in the online supplement Table (Appendix A). We created seven item sets for a total of 74 items: “core” items which focused on the time from the suspicion of cancer through the present (16 items); diagnosis (5 items); deciding about treatment (19 items); surgery (9 items); radiation treatment (10 items); chemotherapy (10 items) and after treatment was concluded (5 items). Items on surgery, radiation and chemotherapy were only administered to respondents who reported they had had that treatment. Responses to these items ranged from never to always, or strongly disagree to strongly agree. Respondents could also choose “does not reply” as an option. Each item set was followed by a single item soliciting an overall rating of communication during the portion of cancer care referenced (response options ranged from poor to excellent). Respondents who provided an overall rating other than excellent were asked to describe where communication “fell short”; responses to the open-ended items will be reported separately.

Additional survey items included questions on cancer diagnosis and treatment, self-rated current health, and a single item on willingness to complete a questionnaire about communication experiences during cancer care. Limited demographic information on respondents was drawn from an existing database described below.

2.2 Sample

The survey sample was selected from a large healthcare system’s existing online member research panel [19]. Panel members were randomly selected from the population of adult patients registered to use the provider’s online patient portal. When patients join the research panel they complete a registration questionnaire and report their demographic characteristics. Once on the panel, patients participate in surveys and other research projects, and receive a small incentive for doing so.

2.3 Survey Administration

A random sample of panelists was selected for this study. Respondents received an email survey that first asked about their cancer history. Only those who reported a cancer diagnosis, and were not employed by the health plan were asked to continue with survey. For the purpose of this study, respondents were not required to have received their cancer care through their current healthcare organization. An incentive valued at $10 was provided. Two reminder emails were sent to non-respondents, the first approximately one week after the initial invitation and the second approximately three weeks later. After approximately six weeks total, the survey was closed, and no additional responses were accepted.

2.4 Analyses

2.4.1 Analysis of “Does Not Apply” Responses

We examined the percentage of respondents choosing Does Not Apply. If a large percentage of respondents selected this response option, it would suggest that the focus of the item was not widely relevant. We also examined whether the percentages of patients choosing Does Not Apply differed for patients reporting non-melanoma skin cancer and those reporting other more serious cancers. We reasoned that some communication goals would be more important to patients with more serious cancers than to patients with non-melanoma skin cancer.

2.4.2 Correlations

We examined the relationship between individual items within an item set and the corresponding overall rating of communication using bivariate correlations and multiple correlations. Bivariate correlations were computed between an individual item referring to a specific aspect of communication within a care phase (e.g. “I was told I had cancer in a way that was sensitive and caring.”) and the corresponding overall rating (e.g., “Overall, how would you rate your experiences with communication when you were diagnosed with cancer?”). Multiple correlations were between the full set of specific communication items for a given phase (e.g., for the five items focused on specific aspects of communication around diagnosis) and the overall rating of communication for that phase. We reasoned that if the individual items within an item set captured the most salient aspects of the patient’s communication experiences during the corresponding phase of care, then the overall rating and the ratings on the specific communication items would be highly correlated, and the items within a set would collectively explain a reasonable percentage of the variance in the corresponding overall rating. For all correlational analyses, Does Not Apply responses were treated as missing.

2.4.3 Reliability of Summary Scores

We computed coefficient alpha for each item set. Coefficient alpha calculated in this way is an appropriate reliability estimate if the focus is the patient (rather than the care team) [20].

2.4.4 Rates of Most Positive Responses and Illustrative Feedback Report

To illustrate how data might be summarized to provide feedback to a team, practice or organization, we computed the percentage of respondents choosing the most positive response (e.g., Always or Strongly Agree) for each item. We chose the most positive response as we considered anything less to be evidence that communication was not fully patient-centered and had room for improvement. We anticipated that patients with more serious cancers have greater communication needs, and that those greater needs would be harder for cancer care teams to meet. Therefore, we compared the proportion of respondents with non-melanoma skin cancer who endorsed the most positive response for each item to the proportion of respondents with more serious cancers. In calculating these percentages, Does Not Apply responses were treated as missing.

3. Results

Survey invitations were sent to a random sample of 7,000 panel members; 2,934 started the survey; 2,334 reported no cancer history; 375 of the remaining 600 members (63%) responded to the full survey. A total of 366 respondents are included in our analyses; eight were omitted because of extensive missing data; one additional respondent was omitted as his/her comments indicated that his/her initial diagnosis of cancer was incorrect.

3.1 Demographic Characteristics of the Sample

Respondent characteristics, including cancer type and time since diagnosis, are summarized in Table 1.

Table 1.

Participant Characteristics

	Overall
	N	%
Total	366	100.0
Gender
Male	161	44.0
Female	205	56.0
Age (in years)
Mean (SD) [range 22–82]	62	(9.4)
Race/Ethnicity
White	306	83.6
Black or African-American	16	4.4
Asian	8	2.2
American Indian or Alaska Native	1	0.3
Multiple racial categories checked	9	2.5
Prefer not to say or no race reported	26	7.1
Hispanic
Yes	17	4.6
No	339	92.6
Prefer not to say	10	2.7
Education
High school graduate, GED, or less	26	7.1
Some college or 2-year degree	113	30.9
4-year college graduate	74	20.2
More than 4-year college degree	153	41.8
Marital Status
Married or living with a partner	257	70.4
Employment Status
Self-employed	53	14.5
Contract worker	12	3.3
Employed outside the home full time	110	30.1
Employed outside the home part time	32	8.7
Unemployed	11	3.0
Student	7	1.9
Homemaker	26	7.1
Retired	163	44.5
Disabled	24	6.6
Other (please specify)	9	2.5
Household income
Under $20,000	14	3.8
$20,000 to $34,999	28	7.7
$35,000 to $49,999	42	11.5
$50,000 to $74,999	62	16.9
$75,000 to $99,999	67	18.3
$100,000 or more	112	30.6
Prefer not to say	41	11.2
Time since diagnosis
Less than 3 months ago	10	2.7
3 to 6 months ago	10	2.7
7 to 11 months ago	11	3.0
1 year to 2 years ago	39	10.7
More than 2 years ago	295	80.6
Missing	1	0.3
Time since treatment
Less than 3 months ago	45	12.3
3 to 6 months ago	22	6.0
7 to 11 months ago	21	5.7
1 year to 2 years ago	49	13.4
More than 2 years ago	216	59.0
Missing	12	3.3
Type of cancer
Breast	78	21.3
Prostate	39	10.7
Colorectal	14	3.8
Lung	3	0.8
Cervical	10	2.7
Skin	116	31.7
Other (non-skin)	91	24.9
Multiple serious cancers	15	4.1
Overall health
Excellent	35	9.6
Very Good	144	39.3
Good	127	34.7
Fair	55	15.0
Poor	5	1.4

3.2 Willingness to Report

Asked about their willingness to report on their communication experiences 32% would have been willing to report after every visit; 45% would be willing to do so after key events, 20% would be willing to report about once a month, and 27% would be willing to report about every six months (respondents could endorse more than one option).

3.3 Analysis of “Does Not Apply” Responses

The number of items within each item set with ten percent or more of respondents marking Does Not Apply by item and by cancer type is provided in Appendix B. Only six of the 74 PACE items were marked Does Not Apply by 10 percent or more of patients with more serious cancer. Of these six items, three focused on help coping with difficulty feelings, and three focused on respect for the patient’s wishes about trying additional treatments.

3.4 Correlational Analyses

Bivariate and multiple correlations between the specific communication items and the corresponding overall rating are presented in Table 2. The bivariate correlations would all be classified as large using Cohen’s conventions [21]. The multiple correlations indicate that between 67 and 87 percent of the variance in the overall rating is accounted for by the specific communication items corresponding to that same period or phase of care.

Table 2.

Correlations and Coefficient Alphas by Item Set

Phase of Cancer Care	Bivariate Correlations (r) (range)	Multiple Correlations (R)	Coefficient Alpha
Entire Continuum	.54 to .76	.87	.96
When Diagnosed	.65 to .73	.82	.90
Treatment Decision Making	.65 to .81	.91	.98
Surgery	.66 to .80	.89	.96
Radiation	.53 to .79	.93	.97
Chemotherapy	.68 to .83	.90	.96
Once Treatment was Completed	.72 to .82	.90	.92

3.5 Reliability of Summary Scores

Coefficient alpha for each item set is .90 or greater (Table 2), suggesting excellent reliability for patient-level scores.

3.6 Rates of Most Positive Responses and Illustrative Feedback

The percentage of respondents choosing the most positive response option (Always or Strongly Agree) ranged from 55 to 84. The items with the lowest percentage of respondents endorsing the most positive response were “I was told I had cancer in a way that was sensitive and caring” during the diagnosis phase and “I had help with difficult feelings, like fear, anxiety and feeling down” once treatment was completed. The items with the highest percentage of patients endorsing the most positive option were “I got the treatment that was best for me” and “My doctor respected my wishes about trying additional treatments” with respect to chemotherapy.

Comparing responses of those with non-melanoma skin cancer to responses of those with more serious cancers revealed statistically significant differences in the percentage of respondents choosing the most positive response on only two items: “I was treated with sensitivity and respect” over the entire course of cancer care, and “I understood why my doctor recommended a certain treatment approach” while deciding about treatment. For both items, fewer respondents with more serious cancer selected Always compared to those with non-melanoma skin cancer.

Figure 1 illustrates how results might be presented for feedback within an organization. We present the percentage of respondents selecting the most positive response, an approach which we have found to be useful in our prior work [21], with a reference line at 75%. Simple graphs such as these can help care teams focus their efforts. For instance, the first graph suggests that communication during the diagnosis phase is a relative weakness, while the second indicates that sensitivity in delivering a cancer diagnosis is an area of particular concern.

Figure 1.

Overall Ratings of Communication

4. Discussion and Conclusion

4.1 Discussion

The six function PCC model provided a conceptual framework for the PACE item set. Items were written with the intent of assessing patients’ perspectives on the six functions across several phases of cancer care. Previously described interviews with patients and family members informed which functions were most relevant at various phase of care, and helped us craft wording that would be understandable to patients [10, 11].

The finding that relatively few patients chose Does Not Apply for the overwhelming majority of items provided evidence of the relevance of the items to patients’ communication experiences. Not surprisingly, we found substantial differences between patients with non-melanoma skin cancer and more serious cancer in rates of selecting Does Not Apply for several items. Most of these items had to do with some aspect of decision-making or emotions. It is very likely that these differences are attributable to the fact that the entire care experience is less emotionally intense for patients with non-melanoma skin cancer, and decision-making is typically less complex. It is also noteworthy that the items for which the largest percentage of patients with serious cancer chose Does Not Apply were related to responding to emotions, consistent with our earlier findings that these issues were seldom introduced by patients during interviews about communication [11], and suggesting that many patients may not be aware that care teams can play a role in providing emotional support during cancer care.

The substantial bivariate and multivariate correlations between the overall rating of communication during a given phase of cancer care and the items assessing the more specific aspects of communication during that phase suggest that the specific items assess aspects of communication that are important to patients.

Coefficient alpha for each item set was excellent. This finding, and the high multivariate correlations, suggest that fewer items could be administered and still yield a relatively stable score, especially for the longer item sets. However, while administering fewer items would reduce respondent burden, doing so would also reduce the amount of information available to providers, teams and organizations. Administering the full set of items will provide the most complete information on strengths and weaknesses and will therefore generate more complete and specific feedback. Thus, while the single “overall” item may provide a reasonable estimate of the percentage of patients who perceive communication to be excellent, the specific items are needed to understand where improvement efforts should be targeted, and assessing progress in those areas. While comments may be useful in deepening users’ understanding of where improvement efforts should be targeted, comments are less useful for gauging progress. Of course, a hybrid approach could also be used, for instance if a clinic were to select and administer a subset of items corresponding to an area of particular concern.

One of the items with the fewest respondents endorsing the most positive response was “I had help with difficult feelings, like fear, anxiety and feeling down” once treatment was completed. The fact that a relatively high proportion of patients gave a suboptimal rating to this item, and to other items related to coping with difficult feelings and with uncertainty suggests that even though some patients may consider these communication goals less important (as evidenced by the rates of those selecting Does Not Apply), many of those who struggle with their emotions, and would accept help from their cancer care team, are not having their needs met. These results are consistent with national data that show “responding to patient emotions” to be one of the more poorly addressed PCC function by healthcare providers [22].

Interestingly, the percentage of patients selecting the most positive option was not significantly different for the two groups examined here (non-melanoma skin cancer versus other more serious cancers), with two exceptions. This consistency in ratings suggests while the salience of some items may be different given severity of illness; amongst those for whom these items are relevant, needs are being met roughly equally across the two groups. Put another way, the areas of greatest unmet need are similar across the two groups, suggesting that cancer care teams face similar challenges communicating with patients with very different diagnoses. This finding also suggests that the items presented here could be appropriate for patients with other diagnoses, with minor modifications.

This study has limitations. First, most respondents were not actively undergoing cancer care, and so were providing responses well after their diagnosis and care. This is clearly a limitation. However, it is likely that patients’ memories of the results of their encounters are more salient than their memories of what happened during those encounters. Second, the characteristics of the sample may limit generalizability of these results, as respondents tended to be well-educated, employed or retired, and all had internet access. Third, additional data and analyses – ideally utilizing generalizability theory – will be needed to appropriately estimate score reliability for team or practice level inferences about scores. Fourth, while we sought to develop items that cover much of the cancer care continuum, we did not develop items focused on long-term survivorship or end-of-life- care. Finally, this study was not designed to examine whether differences in outcomes affect patients’ perceptions of communication. This is an important question for future research.

We hope our findings stimulate researchers to include these items in future studies, and to further evaluate their usefulness. We also hope that cancer care teams and organizations will adopt these items, and report on the usefulness of the resultant feedback. We developed the PACE item sets corresponding to different phases of care to enable longitudinal assessment of patients’ communication experiences; we hope that researchers and care teams will consider longitudinal assessment, and will report on the outcomes.

4.2 Conclusion

Patient-centered communication is an essential component of high quality cancer care. Without good patient-team communication, patients cannot be meaningfully engaged in their care. They cannot make informed decisions, or fully participate in self-management. There is a growing body of evidence on the prevalence of communication breakdowns in cancer care, and the harms that can result. Cancer care teams seeking to improve their ability to communicate effectively with patients may benefit from frequent and specific feedback from patients. The PACE items tap relevant, important aspects of communication during cancer care, and may be useful to cancer care teams desiring detailed feedback. The PACE measure is offered as a tool to support such efforts.

4.3 Practice Implications

The PACE is a new tool designed to help practitioners solicit feedback from patients, and thereby help to prioritize targets for improvement. Health care organizations could use these items to monitoring the performance of multiple teams, evaluate the impact of interventions intended to improve communication, and establish internal benchmarks. The entire set of items could be administered, or a subset of items matching organizational priorities could be selected. Questionnaires could be administered at key junctures in care (e.g., within two weeks of diagnosis) or periodically (e.g., monthly). Implementation details are likely to vary depending on the purpose of the assessment, organizational goals, and practical constraints. While items were administered via the internet for this study, paper-based administration is also possible, and may be desirable in some settings, e.g., with patients who do not have internet access, or are uncomfortable using computers. The PACE items are freely available for use from the first author, and are included in Appendix A. Organizations and teams who register can provide an online link to their patients, and will receive monthly automated reports summarizing results for patients who indicate receiving care with that team (pccfs.org).

Figure 2.

Ratings of Communication at Diagnosis

Practice Implications.

The PACE is a new tool for eliciting patients’ perspectives on communication during cancer care. It is freely available online for practitioners, researchers and others.