Was a decision made? An assessment of patient–clinician discordance in medical oncology encounters

Abstract

Background

Shared decision making is promoted to improve cancer care quality. Patients and clinicians may have different ideas about what constitutes a cancer care decision, which may limit the validity of self‐reported measures of shared decision making. We sought to estimate the extent to which patients and clinicians agree on whether a cancer care decision was made during an outpatient encounter.

Methods

We surveyed patients and clinicians immediately after an oncology encounter at a large, tertiary medical centre and calculated agreement in response to the single‐item question, ‘Was a specific decision about cancer care made during the appointment today?’ Answer options were ‘yes’ and ‘no’. Participants were 315 oncology patients, with any solid tumour malignancy and at any stage of management, and their clinicians (22 staff oncologists, nine senior fellows and five nurse practitioners).

Results

Patients and clinicians reported having made a cancer care decision in 184 (58%) and 174 (55%) of encounters, respectively. They agreed on whether a cancer care decision was made in 213 (68%) of encounters (chance‐adjusted agreement was 0.34); in 56 of the 102 discordant encounters, the patient reported making a decision while the clinician did not. We found no significant correlates with discordance.

Conclusions

Patients and clinicians do not always agree on whether a cancer care decision was made. As such, measures that ask patients and/or clinicians to evaluate a decision‐making process or outcome may be methodologically insufficient when they do not explicitly orient respondents towards the thing they are being asked to assess.

Background

Introduction

Patient‐centred care requires optimal com‐munication of research evidence and shared decision‐making (SDM) between clinicians and patients. SDM respects patients, while enabling them to incorporate clinical evidence and expertise into their choices. While the ethical imperative in support of SDM is strong and sufficient,1 SDM interventions can sometimes improve healthcare quality and reduce healthcare utilization.2, 3 These factors have led to legislation and other efforts to widely implement SDM into practice.4, 5 Evaluating the success of such efforts, however, requires a reliable means of assessing when and with what quality SDM is actually occurring.2, 6, 7, 8, 9, 10

Medical oncology encounters are characterized by complexity and gravity. In addition to deliberations about standard treatments, patients and clinicians may need to address topics such as alternative therapies, end of life issues and psychosocial concerns. While some of these decisions would be considered technical, many are not, and thus, patients have a legitimate interest in ensuring the course of action is consistent with their values, preferences and goals. Unfortunately, oncology encounters may be particularly susceptible to communication failures that can impede optimal SDM activity.11, 12, 13, 14, 15, 16, 17, 18

To promote and evaluate SDM in clinical practice, research has focused on the development of decision aids and metrics for the decision‐making process.19, 20, 21, 22, 23, 24, 25, 26, 27 SDM measures often assess different constructs and vary by who evaluates the interaction.28, 29 Metrics may be designed to assess decisional quality from the perspective of a third‐party observer,22, 26 or the participants themselves.21 For participants to assess a decision‐making process, they must first be able to recognize when they have participated in one, yet no literature supports this key assumption as it applies to oncology encounters.

Objective

We sought to determine the extent to which medical oncology patients and clinicians agreed in their ratings of whether a specific cancer care decision had been made during an oncology encounter. To generate hypotheses about the causes of discordance in this area, we also searched for the presence of associated correlates.

Methods

Participants and design

The present analysis is part of an on‐going, multicenter parent study assessing cancer communication through the use of audio recordings and paper‐based surveys.30 Participants include patients, at any stage of treatment for any solid tumour malignancy, and their clinicians (staff oncologists, oncology fellows and nurse practitioners). For this analysis, we assessed the baseline characteristics and immediate post‐visit impressions of all eligible participants (315 patients and 36 clinicians) that enrolled at the Mayo Clinic site. Mayo Clinic is a large, tertiary medical centre in the Upper Midwest region of the United States. The Institutional Review Boards of Mayo Clinic and all partner sites approved this study.

Measures and procedure

A study coordinator screened all participants for eligibility, approaching and consenting patients at regularly scheduled oncology visits. Upon enrolment, the study coordinator administered a pre‐encounter questionnaire to collect baseline and demographic information from all participants (i.e. patients and clinicians). Patients completed these surveys in their examination rooms after giving consent and just prior to their clinical encounters; clinicians completed them prior to their first study encounter.

Immediately after the encounter, and while still in the examination room, patients completed a post‐encounter questionnaire that assessed their perceptions of the encounter's communication quality and the content of the discussion, as well as their overall evaluation of the clinician. This survey included a total of 25 items and included three scales assessing different aspects of communication quality. One of these scales, the SDMQ‐9, is a validated, 9‐item measure of a patient's assessment of shared decision‐making activity.21 Because we did not expect decisions to be made in all encounters, we included a screening question to ensure patients completed the SDMQ‐9 only if they felt a specific cancer care decision was made during the encounter. In all but a few cases (in which the patient felt too ill or tired), patients returned their pre‐ and post‐encounter surveys to the study coordinator before leaving the clinic.

Clinicians also completed a brief, 8‐item questionnaire immediately after these same encounters. This survey queried clinicians about the patient's point on the cancer care continuum (across six stages ranging from initial diagnosis to end‐stage disease), the perceived quality of the encounter and the perceived rapport with the patient. It also asked clinicians to respond to the same SDMQ‐9 screening question that was administered to patients (i.e. was a specific cancer care decision made?), although a clinician version of the SDMQ‐9 was not administered. Clinicians typically returned their completed questionnaires to the study coordinator before seeing their next patient.

Thus, as part of the post‐encounter surveys administered to patients and clinicians, we asked the common question: ‘Was a specific decision about cancer care made during the appointment today?’ Answer options were ‘yes’ and ‘no’. This afforded us the unique opportunity to obtain immediate and reliable patient and clinician perceptions of whether a decision had occurred within the same clinical encounter.

All surveys were pre‐tested through cognitive interviews and within a pilot study of representative patients (n = 15) and clinicians (n = 3) from the same oncology unit. At no point did a clinician or patient express confusion about what the key question was asking or how best to answer it. After surveys were collected, data were double‐entered into a secure, web‐based data management system (REDCap31) and merged prior to analysis.

Data analysis

For our primary analysis, we classified all encounters as concordant or discordant based on patient and clinician responses to the central question of whether a decision was made and categorized the latter further by who responded ‘yes’. We also evaluated agreement in decision identification beyond that expected by chance alone by calculating the kappa statistic32 for each clinician‐determined cluster (thus generating a set of clinician‐specific kappa values), and then weighted overall by the number of encounters per clinician (a weighted average kappa value). We also explored whether the concordance outcome was influenced by a clustering effect by determining the intracluster correlation coefficient (ICC).

For our exploratory analyses, we used bivariate tests of association to identify patient and clinician characteristics correlated with discordance. The characteristics tested were chosen based on the data we had available, our own hypotheses about potential mediating variables for discordance, and whether they had been previously studied in regard to their effects on patient–clinician communication and/or SDM.33, 34, 35, 36, 37, 38 These included patient‐reported demographics (age, sex, race, education, income and tumour type), health literacy (comfort level with spoken and written health information39), encounter satisfaction, clinician ratings (on a scale of 1–10) and assessments of communication quality (whether the clinician explained things in an understandable way, listened carefully, addressed all concerns and involved the patient in deciding what to discuss). We also examined whether clinician age, sex, type (oncologist, fellow, or nurse practitioner), encounter satisfaction or perception of rapport were associated with discordance. To provide a crude estimate of the influence of the potential type of decision made on its association with discordance, we used the clinician‐reported assessment of the patient's point on the cancer care continuum (i.e. initial diagnosis vs early initial treatment vs mid‐initial treatment vs survivorship/remission vs recurrence vs end‐stage disease) (this analysis assumed that patients being seen at initial diagnosis would be making different types of decisions from those in remission, for example). We evaluated all continuous data with two‐sided t‐tests, and categorical items were assessed with chi‐square tests for independence or Fisher's exact test as appropriate. Significance was established as P < 0.05. To assess for the potential of indiscriminate and routinized clinician ratings of ‘whether a decision was made,’ we looked for patterns of consistent, within‐clinician responses to this question (this analysis was pursued to ensure that clinicians were making thoughtful judgments and not simply checking the same box for all encounters).

Results

We excluded 12 encounters from the analysis because the patient and/or clinician did not answer the survey question of interest. Table 1 summarizes the characteristics of the patients and clinicians participating in the 315 eligible encounters.

Table 1.

Characteristics of enrolled patients and clinicians

Patient characteristics (n = 315)
Male (%)	129 (41.0)
Age (SD)	61.5 (13.3)
Race (%)
White	304 (96.8)
Other	10 (3.1)
Education (%)
High school or less	92 (29.7)
Some college	88 (28.4)
College graduate	54 (17.4)
Postgraduate	74 (23.9)
Household income (%)
<$20 000	22 (7.4)
$20–50 000	102 (34.7)
$50–75 000	44 (15.0)
>$75 000	126 (42.9)
Tumour type (%)
Brain	10 (3.2)
Breast	93 (29.5)
Endocrine	1 (0.3)
Gastrointestinal	72 (22.9)
Genitourinary	26 (8.3)
Gynecologic	25 (7.9)
Head/Neck	11 (3.5)
Lung	33 (10.5)
Melanoma	21 (6.7)
Sarcoma	20 (6.3)
Other	1 (0.3)
Cancer care continuum (%)
Initial diagnosis	14 (4.4)
Early initial treatment	29 (9.2)
Mid‐initial treatment	61 (19.4)
Remission	103 (32.7)
Recurrence	93 (29.5)
End‐stage	15 (4.8)
Clinician characteristics (n = 36)
Staff oncologists (n = 22)
Male (%)	16 (72.7)
Age (SD)	47.5 (9.94)
Oncology fellows (n = 9)
Male (%)	9 (100)
Age (SD)	33.7 (2.1)
Nurse practitioners (n = 5)
Male (%)	0 (0)
Age (SD)	37.6 (7.5)

Patients and clinicians agreed on whether a decision had taken place in 213 (68%) of the encounters, and the kappa statistic was 0.34 (SE = 0.5) when assuming independence (range across clinicians was −0.5 to 1.0). Clinicians participated in an average of 8.75 encounters (range 6–12). The cluster size‐weighted average of the kappa statistic was 0.33. The ICC was 0.04 (95% CI 0–0.11), indicating that there was no significant effect due to clustering on the findings and that there was no need for an adjusted analysis. In assessing whether a decision was made, no clinician chose the same response for all encounters. 32 (94%) chose ‘yes’ and ‘no’ with equal frequency ±30% and 15 (42%) chose ‘yes’ and ‘no’ with equal frequency ±10%. In 56 of the 102 discordant encounters, the patient reported having made a decision when the clinician had not (Fig. 1). We could not identify any significant correlates with discordance (data available upon request).

Figure 1.

Patient–clinician agreement on whether a cancer care decision was made in an outpatient oncology encounter.

Discussion

When completing a survey immediately following a clinical encounter, patients and clinicians sometimes disagree in their report of whether they made a cancer care decision during that encounter. This finding adds a novel aspect to the literature on differences between the experiences of patients and clinicians during oncology visits.11, 12, 40 The cause for this discordance is not known, and our exploratory analyses were unable to shed additional light in this regard.

While the most direct explanation for the discordance might be general communication challenges during oncology visits, two possibilities deserve greater scrutiny. First, discordance in decision identification may derive from discordance in decision definition. To our knowledge, no accepted definition for what constitutes a ‘cancer care decision’ exists in the literature. Such a definition may be strongly shaped by training, values and/or healthcare beliefs. For example, patients may perceive a question like ‘Was a decision made?’ to be synonymous with ‘Did you come to some conclusion or achieve closure today?’, whereas physicians may interpret it to ask the more exclusive question of ‘Did you undertake a decision‐making process?’ Our data suggests that clinicians are discriminant in their assessment of whether a decision was made, but it is unclear what criteria they use when making this determination. Whether establishing a standardized definition for or description of what constitutes a cancer care or medical decision, particularly for use in self‐reported decision‐making process measures, would be feasible or of any practical value is unclear.

Discordance might also result from problems in communication, whereby clinicians and/or patients make decisions that are not labelled as such and/or not disclosed to the other party. During the visit, for example, the clinician might ‘decide’ to move to the next step in a protocol, but this is not typically labelled as a decision, and the patient might feel this is just the next logical or technical step in a journey that was agreed upon long before. Additionally, paternalistic decision making with minimal patient participation might result in decisions that are not apparent to the patient. To the extent that this is the case, instances of discordance might indicate a lack of SDM. This idea is considered in some SDM assessment tools, including the OPTION score.41 Our analysis suggests that patients may also make undisclosed decisions during clinical encounters, a topic that has received little consideration in the SDM field. This might be explained by differences in encounter expectations and/or agendas among patients and clinicians. Such differences could affect each participant's sensitivity to and awareness of the types of discussions and/or decisions that take place.

Limitations

Certainly, our study has a number of limitations and its findings should be confirmed in larger investigations and in settings other than oncology. Our study relied strictly on cross‐sectional survey data in a demographically homogenous population at one centre. To better assess the importance of concordance/discordance in decision identification, longi‐tudinal outcome data will be required and analysis of the clinical impact will need to be documented. This study was limited by evaluations based on a single survey item and should be viewed as hypothesis‐generating. We restricted item responses to ‘yes’ and ‘no’, but the addition of an ‘unsure’ category might have been informative. We also did not ask patients or clinicians to identify the specific decision they felt was made. Thus, our analysis only considers the chance‐adjusted agreement in ratings of the very inclusive question of whether any decision was made (i.e. patients and clinicians might agree that a decision was made, but be referencing entirely different decisions). As such, the low agreement we find in this analysis represents a ‘best‐case’ scenario of the extent to which patients and clinicians agree in their ratings of whether the same decision was made. Future study on this topic should ask this additional question of both parties while triangulating with third‐party observers and characterizing and comparing the types of decisions each identify and rate. Qualitative methods would likely provide additional insight.

Practice implications

This study adds to the literature on patient and clinician discordance in oncology and suggests opportunities for improvement in communication. Further work is also needed to define what constitutes a cancer care decision, to decide whose assessment matters most in making this determination and to know how best to integrate the assessments of participants with those of third‐party observers. Importantly, assessors will also need to determine whether the ‘quality’ of SDM is a reflection of the process (by means of information sharing, deliberation, consensus), or of its outcomes (whether or not a mutually identifiable end was reached).

Conclusion

Arguably, the perception of a single stakeholder should not be considered sufficient for establishing whether or not a cancer care decision has been made, especially if the goal is to measure shared decision making.7, 28 Self‐reported SDM metrics recognize the surveyed stakeholder alone as the arbiter of whether a decision was made. In so doing, these metrics pre‐suppose that whether a decision was made is a verifiable fact, accessible to all present – an assumption that our findings challenge.

Our results suggest that the joint work of making decisions in health care is often assessed and reported differently by clinicians and patients. This creates challenges in our ability to determine whether a decision took place, let alone whether the process used to make it was of high quality.

Author contributions

AL conceptualized the study, recorded and analysed the data, drafted the manuscript and approved the final manuscript. KH conceptualized the study, analysed the data, assisted in drafting the manuscript and approved the final manuscript. CF assisted in recording and displaying data and in drafting the manuscript, provided substantive feedback and approved the final manuscript. VM assisted with interpretation of the data and in drafting and revising the manuscript, provided oversight in data analysis and approved the final manuscript. KY assisted with interpretation of the data and contributed to the analytic approach, critically revised the manuscript and approved the final version. AK contributed to the conceptualization of the study, critically revised the manuscript and approved the final version. GG contributed to the conceptualization of the study, critically revised the manuscript and approved the final version. JT conceptualized the study, provided oversight for the analytic approach, contributed to the drafting and revision of the manuscript and approved the final version.

Competing interests

The authors declare that they have no competing interests.

Funding

The parent study for this analysis is funded by a grant from the National Center for Complementary and Alternative Medicine (NCAAM), Grant # R01AT006515. This publication was also made possible by CTSA Grant Number UL1 TR000135 from the National Center for Advancing Translational Sciences (NCATS), a component of the National Institutes of Health (NIH). Its contents are solely the res‐ponsibility of the authors and do not necessarily represent the official view of NIH. The funding source did not participate in any part of this study.