Abstract
Background
Overuse of surveillance testing for breast cancer survivors is an important problem but its extent and determinants are incompletely understood. The objectives of this study were to determine the extent to which physicians’ breast cancer surveillance testing beliefs are consistent with test overuse, and to identify factors associated with these beliefs.
Methods
2009–2010 cross-sectional survey of US medical oncologists and primary care physicians (PCPs). Physicians responded to a clinical vignette ascertaining beliefs about appropriate breast cancer surveillance testing. Multivariable analyses examined the extent to which test beliefs were consistent with overuse and associated with physician and practice characteristics and physician perceptions, attitudes, and practices.
Results
1098 medical oncologists and 980 PCPs completed the survey (response rate 57.5%). Eighty-four percent of PCPs (95% CI: 81.4%–86.5%) and 72% of oncologists (95% CI: 69.8%– 74.7%) reported beliefs consistent with blood test overuse, while 50% of PCPs (95% CI: 47.3%– 53.8%) and 27% of oncologists (95% CI: 23.9%–29.3%) reported beliefs consistent with imaging test overuse. Among PCPs, factors associated with these beliefs included smaller practice size, lower patient volume, and practice ownership. Among oncologists, factors included older age, international medical graduate status, lower self-efficacy (confidence in knowledge), and greater perceptions of ambiguity (conflicting expert recommendations) regarding survivorship care.
Conclusions
Beliefs consistent with breast cancer surveillance test overuse are common, greater for PCPs and blood tests than for oncologists and imaging tests, and associated with practice characteristics and perceived self-efficacy and ambiguity about testing. These results suggest modifiable targets for efforts to reduce surveillance test overuse.
Introduction
Cancer surveillance testing is a critical yet problematic component of follow-up care for breast cancer survivors who have completed active treatment. The high risk of disease recurrence in these patients provides justification for early detection efforts, and several laboratory and imaging tests are often used by physicians for this purpose. These include blood tests (e.g., complete blood count (CBC), liver function tests (LFTs), serum tumor markers), and imaging examinations (e.g., chest x-ray (CXR), advanced diagnostic imaging (ADI) studies including bone, computed tomography (CT), and magnetic resonance imaging (MRI) scans) to detect recurrent or metastatic disease.3–5 However, of all these surveillance tests only mammography is supported by evidence and recommended in clinical practice guidelines.1,2 Consequently, in their recent “Choosing Wisely” campaign, the American Society of Clinical Oncology and the American College of Physicians identified non-mammographic breast cancer surveillance testing as an overused, unnecessary intervention that physicians and patients should question.9–12
Overuse of unnecessary health services is a significant problem,8,11,13,14 and overuse of breast cancer surveillance testing poses particular clinical and economic challenges. The population of cancer survivors is rapidly growing, increasing the demand for surveillance testing and the potential impact of test overuse.15 In 2007, there were 11.7 million cancer survivors in the US—of which breast cancer survivors represented the largest group (22%)— and their numbers continue to expand.16 Yet growth in the oncology workforce is not keeping pace, raising a need for other providers including PCPs to play a more active role in cancer survivor care. The Institute of Medicine (IOM) has thus recommended “PCP-centered” or “shared care” models as alternatives to the current “oncologist-centered” model of cancer survivor care.17 This may be a rational response; however, it complicates care delivery and could thus contribute to cancer surveillance testing overuse. Such overuse, furthermore, has downstream consequences due to “cascade” effects, in which unnecessary testing leads to clinical interventions that, in turn, result in adverse clinical outcomes and added health care costs.18,19
These issues underscore the importance of examining overuse of unnecessary, non-guideline-recommended breast cancer surveillance testing among oncologists and PCPs. This problem has been explored in population-based cohort studies using administrative data.3,4,20 However, these studies have had limited ability to distinguish the indication for testing (surveillance vs. diagnostic evaluation), or to account for variation in patient characteristics that may influence recurrence risk and the appropriate level of surveillance monitoring. An alternative methodology that overcomes these limitations is the use of surveys employing hypothetical clinical vignettes to ascertain physicians’ beliefs about appropriate testing or treatment.21–23 By standardizing patient characteristics, clinical vignettes provide an “inherently case-mix adjusted,” physician-specific measure of practice intensity22,23 that corresponds well to physician practice.24–26 Vignettes have begun to be employed in studies of cancer surveillance testing.5,27
We recently conducted a study of vignette data from a nationally-representative physician survey, and found that 79%of PCPs and 69% of medical oncologists believed in using at least one blood test, while 42% of PCPs and 22% of oncologists believed in using at least one imaging test other than mammography for breast cancer surveillance.28 However, this study did not analyze the total number of different tests endorsed by physicians, or the factors associated with test overuse beliefs. The objectives of the current study were to extend this prior work by: 1) more fully describing the extent to which PCPs’ and oncologists’ beliefs about breast cancer surveillance testing are consistent with test overuse; and 2) identifying physician and practice characteristics, perceptions, and attitudes associated with these beliefs. Our overarching goal was to identify potential causes of surveillance test overuse among PCPs and oncologists, and modifiable targets for efforts to reduce overuse.
Methods
Data source and study population
We used data from the Survey of Physician Attitudes Regarding the Care of Cancer Survivors (SPARCCS), a nationwide survey of medical oncologists and PCPs conducted by the National Cancer Institute (NCI) and the American Cancer Society (ACS) in 2009–2010. SPARCCS was designed to assess medical oncologists’ and PCPs’ beliefs, attitudes, knowledge, and practices regarding follow-up care of breast and colorectal cancer survivors. SPARCCS utilized two complementary questionnaires—one for PCPs, the other for oncologists—designed to allow comparison of these physicians. SPARCCS items were adapted from existing instruments29–34 and developed by our interdisciplinary team.
A nationally-representative sample of actively-practicing, office-based PCPs and medical oncologists who reported continued care of breast cancer patients was drawn from the American Medical Association Physician Masterfile. Questionnaires were mailed to 3596 eligible physicians, and 2202 (1072 PCPs and 1130 oncologists) returned completed questionnaires; the absolute response rate, calculated using the standard formula RR3,35 was 57.5% and the cooperation rate—which excludes physicians for whom we did not have valid contact information—was 65.1%. For the current analysis, we further excluded 51 PCPs who reported not treating breast cancer patients since completing training. Additional methodological details are reported elsewhere,28 and survey instruments are available at http://healthservices.cancer.gov/surveys/sparccs/. The study and survey instruments were approved by the NCI IRB and the US Office of Management and Budget.
Variables and measures
Although overuse of health services has been a major focus of research and policy discussions,6,8,10,13,36,37 a single conceptual framework outlining its causes is lacking. We therefore selected variables hypothesized or demonstrated in past research to be associated with overuse. Overuse has been conceptualized as a manifestation of “ clinical waste” reflecting the production of services providing marginal or no benefit to patients,38,39 and arising from system-level factors including fee-for-service incentives and health information systems that influence the supply and efficiency of health services.38,40,41 Medicolegal risk has been identified as another important determinant.7,13,21,39,42,43 Studies of laboratory and imaging test ordering have shown that overuse is associated with physician characteristics including age, sex, and country of training;21,44 practice characteristics including size and HMO and hospital affiliation, which may relate to various factors including accessibility of clinical information and standardization of care;21 and physician perceptions of self-efficacy (confidence regarding one’s own ability to perform a behavior)45–47 and ambiguity (perceived lack of strength or consistency) of scientific evidence or expert recommendations.45,48 Perceived ambiguity can cause avoidance of decision making—a response known as “ambiguity aversion”49—and could thereby lower utilization of health services by physicians. Finally, specific cancer survivorship care practices may influence surveillance test overuse. Use of survivorship care plans and oncologist-PCP communication about mutual care roles and responsibilities may facilitate guideline-consistent, coordinated cancer survivor care, and thereby reduce surveillance test overuse.
Beliefs about cancer surveillance testing
The main dependent variable was physicians’ beliefs about appropriate laboratory (CBC, LFTs, serum tumor markers) and imaging tests (CXR, ADI including CT, MRI, and bone scans) for breast cancer surveillance. We presented PCPs and oncologists with a hypothetical clinical vignette describing a breast cancer patient at low recurrence risk: “How often do you believe the following cancer surveillance tests should be performed for a breast cancer survivor with the following characteristics: 55-year old woman, status post adjuvant chemotherapy for early stage breast cancer four years ago, currently asymptomatic, no evident disease, no significant co-morbidities, not on endocrine therapy?” We then presented a list of test options and intervals (“ every 3–4 months”/“ 6 months”/“yearly” for laboratory tests, and “ yearly”/“ every 2–3 years”/“every 4–5 years” for imaging tests), and “ only if indicated,” “ never,” “ don’t know,” and “ other.” This single-item vignette was developed by our multidisciplinary expert team; space limitations prevented use of multiple vignettes. Cognitive interviewing with a convenience sample of 9 PCPs and 9 oncologists was used to test the item’s comprehensibility and to refine the vignette.
We defined “ overuse” based on clinical practice guidelines issued by the American Society of Clinical Oncology and the National Comprehensive Cancer Network.1,2 Although strong negative evidence on the comparative effectiveness of non-mammographic breast cancer surveillance tesings is lacking, there is expert consensus that non-mammographic testing constitutes overuse and should be decreased. Neither guideline recommends routine use of any laboratory or imaging tests other than annual mammography for early-stage breast cancer survivors. We therefore classified responses other than “ only if indicated,” “ never,” “ don’t know,” or “ other” as overuse. Including “ don’t know” and “ other” responses in the non-overuse category results in a bias towards underestimating beliefs in test overuse, but represents the most conservative approach to the data. We did not include mammography since 98% of PCPs and 99% of oncologists correctly recommended annual mammography. We created separate overuse belief variables for laboratory and imaging tests, hypothesizing that these tests may be perceived differently and have distinct determinants due to their differing informational value, benefits, harms, and costs. We created variables to assess the extent to which physicians’ beliefs reflected overuse by summing recommended tests of each type. We created 3 ordinal response categories based on the distribution of overused tests: 0 tests/1–2 tests/3 tests for laboratory tests, and 0 tests/CXR/CXR±ADI for imaging tests. We combined responses of 1 and 2 blood tests to avoid problems of small cell sizes in multivariate analyses, since only 6.7% of oncologists and 7.7% of PCPs endorsed just one test.
Physician and practice characteristics
Physician characteristics included age, sex, race, and US vs. international medical graduate status. Practice characteristics included Metropolitan Statistical Area (MSA) location, practice size, employment arrangement (full/part owner, employee of physician-owned vs. large group/ HMO vs. academic/university practice), teaching hospital affiliation, electronic medical record (EMR) use, and proportion of uninsured patients.
Physician perceptions, attitudes, and practices
Self-efficacy regarding cancer surveillance testing was measured by an item assessing physicians’ confidence in their knowledge: “ How confident do you feel about your knowledge of the following aspects of cancer-related follow-up care for breast cancer survivors?” We analyzed responses to the sub-item: “ Appropriate surveillance testing to detect recurrent cancer”; response options were “ not at all confident,” “ somewhat confident,” and “ very confident.”
Perceived ambiguity about expert recommendations for cancer survivor care was measured by the item, “ I believe there are conflicting recommendations regarding the appropriate management of cancer survivors who have completed active treatment for early stage breast cancer,” with a 4-point response scale from “ strongly disagree” to “ strongly agree.”
Use of cancer survivorship care plans was ascertained by an item asking physicians how often they receive (PCPs) or provide (oncologists) an explicit follow-up care plan documenting recommendations for future care and surveillance from or to the other physician, respectively. Role communication was ascertained by a composite variable (α=.86) that averaged responses to 4 questions scored on a 5-point response scale, ranging from “ never” to “ always/almost always.” Two questions asked how often physicians routinely communicate with their patients about what physician will follow them for 1) “ their cancer” and 2) “ other medical issues,” while 2 analogous questions asked about communication with other physicians.
Defensive medicine was measured by an item asking physicians how often they “ order tests or treatments to protect against malpractice litigation,” using the same response scale. PCPs’ involvement in cancer surveillance testing was ascertained by asking PCPs how screening for recurrent breast cancer is usually delivered for breast cancer survivors in their practice.
Response options were: “ I order or provide this service myself,” “ the oncology specialist orders or provides this service,” “ the oncology specialist and I share responsibility for ordering or providing this service,” “ another specialist orders or provides this service,” or “ I am not involved in this care.” Based on the distribution of responses, we grouped PCPs into three categories: 1) provides (“ I order or provide this service myself”); 2) co-manages (“ the oncology specialist and I share responsibility for ordering or providing this service”); or 3) not directly involved (“ the oncology specialist orders or provides this service,” “ another specialist orders or provides this service,” or “ I am not involved in this care”). Notably, 43% of PCPs reported no involvement in breast cancer surveillance testing. We therefore conducted sensitivity analyses excluding these physicians (described below), to explore whether PCPs’ lack of involvement in test decisions moderated any of the associations examined in the study.
Data analysis
We computed descriptive statistics on physician and practice characteristics and surveillance testing beliefs for PCPs and oncologists. We then conducted multivariable polytomous logistic regression analyses with surveillance testing beliefs as the dependent variable, including all independent variables in full-fitted models. We fitted separate models for PCPs and oncologists, and for blood and imaging tests. All analyses were conducted using the statistical program SUDAAN. Survey weights adjusting for undercoverage and survey nonresponse were applied in the analyses; the weighted data yield national estimates.
Results
Descriptive data
The final analytic sample consisted of 2078 physicians (980 PCPs, 1098 oncologists), excluding 41 PCPs and 32 oncologists with missing data on any dependent variable. Sample population characteristics are in Table 1.
Table 1.
PCPs (N=980) |
Oncologists (N=1098) |
||||
---|---|---|---|---|---|
Na | % | Na | % | ||
Physician characteristics | |||||
Age | |||||
< 40 years old | 226 | 22.7 | 320 | 30.9 | |
40–49 years old | 324 | 33.8 | 293 | 28.9 | |
50–59 years old | 293 | 30.4 | 290 | 24.7 | |
60+ years old | 137 | 13.2 | 195 | 15.5 | |
Sex | |||||
Female | 330 | 36.1 | 285 | 27.2 | |
Male | 650 | 63.9 | 813 | 72.8 | |
Race | |||||
NH White | 686 | 70.9 | 710 | 63.1 | |
Asian | 165 | 15.0 | 289 | 28.1 | |
Other | 107 | 11.7 | 76 | 6.9 | |
Unknown | 22 | 2.3 | 23 | 2.0 | |
Income | |||||
Not salaried | 353 | 36.2 | 266 | 23.2 | |
Primarily fee-for-service | 388 | 40.2 | 465 | 41.7 | |
Primarily capitation | 134 | 13.0 | 118 | 11.4 | |
Salaried, not specified | 104 | 10.5 | 244 | 23.4 | |
Unknown | 1 | 0.1 | 5 | 0.4 | |
Country of training | |||||
U.S. | 725 | 76.7 | 739 | 64.6 | |
International | 255 | 23.3 | 359 | 35.4 | |
Practice characteristics | |||||
Volume of breast cancer patients (PCP/Oncologists) | |||||
0–5/year; 0–10/week | 238 | 24.4 | 343 | 31.8 | |
6–15/year; 11–20/week | 288 | 28.9 | 363 | 33.1 | |
16–25/year; 21–30/week | 181 | 19.6 | 204 | 17.9 | |
26+/year; 31+/week | 252 | 25.0 | 184 | 16.8 | |
Unknown | 21 | 2.0 | 4 | 0.4 | |
Practice size (number of physicians) | |||||
1 | 240 | 23.9 | 119 | 10.1 | |
2–5 | 408 | 43.1 | 422 | 39.1 | |
6–100+ | 316 | 31.4 | 538 | 49.0 | |
Unknown | 16 | 1.6 | 19 | 1.8 | |
Employment arrangement | |||||
Full/Part owner of physician practice | 526 | 54.8 | 512 | 45.0 | |
Employee of physician owned practice | 94 | 10.4 | 116 | 11.3 | |
Employee of large group/HMO | 182 | 17.5 | 113 | 10.6 | |
Employee of university hospital or clinic | 62 | 6.3 | 234 | 21.6 | |
Other | 111 | 10.7 | 112 | 10.1 | |
Unknown | 5 | 0.5 | 11 | 1.4 | |
EMR use | |||||
Paper records | 373 | 39.9 | 241 | 21.8 | |
Partial EMR | 124 | 13.2 | 207 | 19.0 | |
In transition from paper | 154 | 14.9 | 277 | 24.5 | |
Full EMR | 315 | 30.8 | 363 | 33.4 | |
Unknown | 14 | 1.3 | 10 | 1.3 | |
Proportion of uninsured patients | |||||
0–5% | 619 | 64.1 | 740 | 66.7 | |
6–25% | 273 | 27.4 | 231 | 20.6 | |
26–100% | 46 | 4.5 | 44 | 4.2 | |
Unknown | 42 | 4.0 | 83 | 8.5 | |
Teaching hospital affiliation | |||||
Yes | 394 | 41.1 | 569 | 51.9 | |
No | 580 | 58.4 | 527 | 48.0 | |
Unknown | 6 | 0.5 | 2 | 0.2 | |
Metropolitan Statistical Area (MSA) | |||||
≥ 1 million | 597 | 61.4 | 709 | 65.8 | |
All other MSAs | 383 | 38.6 | 389 | 34.2 | |
Physician attitudes and practices | |||||
PCP involvement in breast cancer surveillance testing (PCPs only) | |||||
Not Involved | 397 | 43.0 | |||
Co-manages (shared care) | 396 | 38.6 | |||
Provides | 81 | 17.7 | |||
Unknown | 6 | 0.7 | |||
Self-efficacy (confidence in knowledge of breast cancer surveillance testing) | |||||
Not at all confident | 55 | 6.4 | 5 | 0.5 | |
Somewhat confident | 506 | 51.8 | 157 | 14.6 | |
Very confident | 413 | 41.1 | 935 | 84.8 | |
Unknown | 6 | 0.7 | 1 | 0.1 | |
Perceived ambiguity about surveillance testing recommendationsb | |||||
Disagree | 179 | 19.2 | 377 | 34.6 | |
Don't know | 119 | 12.5 | 21 | 1.9 | |
Agree | 651 | 65.1 | 674 | 61.2 | |
Unknown | 31 | 3.2 | 26 | 2.3 | |
Defensive medicine (test ordering to protect against malpractice litigation) | |||||
Never | 133 | 14.2 | 202 | 18.9 | |
Rarely | 317 | 31.9 | 445 | 40.0 | |
Sometimes | 331 | 33.3 | 325 | 29.7 | |
Often | 130 | 13.3 | 95 | 8.6 | |
Almost Always | 25 | 2.4 | 11 | 1.0 | |
Unknown | 44 | 4.9 | 20 | 1.9 | |
Use of breast cancer survivorship care plans | |||||
Never | 118 | 12.0 | 43 | 3.9 | |
Rarely | 242 | 24.6 | 181 | 16.4 | |
Sometimes | 249 | 25.2 | 275 | 24.8 | |
Often | 206 | 21.1 | 306 | 27.4 | |
Almost Always | 152 | 15.8 | 271 | 25.0 | |
Unknown | 13 | 1.4 | 22 | 2.4 | |
Role Communicationc | |||||
Mean, SE | 3.15 | 0.04 | 3.88 | 0.04 |
N is the unweighted sample size and percents are weighted to the US population of physicians
Perceived ambiguity = belief in conflicting expert recommendations regarding appropriate management of breast cancer survivors
Role communication = aggregate measure (range 1–5) of self-reported extent of communication with patients and other physicians about which physician will follow patients for their cancer and other health problems
Data source: Survey of Physician Attitudes Regarding the Care of Cancer Survivors (SPARCCS)
Beliefs about breast cancer surveillance tests
Figure 1 shows the extent to which PCPs’ and oncologists2019; beliefs about appropriate surveillance testing reflected test overuse. A majority of physicians reported beliefs consistent with blood test overuse, although the proportion was greater for PCPs (84%, 95% CI: 81.4%–86.5%) than oncologists (72%, 95% CI: 69.8%–74.7%). Overuse beliefs were less common for imaging tests, although 50% (47.3%–53.8%) of PCPs recommended at least 1 non-indicated test, compared with 27% (23.9%–29.3%) of oncologists. This difference was attributable to fewer oncologists (8%) than PCPs (31%) endorsing beliefs consistent with ADI+CXR overuse, vs. CXR overuse alone.
Factors associated with test overuse beliefs
Table 2 shows the factors associated in multivariable analyses with test overuse beliefs among PCPs (only significant associations (p<.05) are shown). Blood test overuse beliefs were inversely associated with larger practice size and employment in a large group/HMO or “ other” practice type (non-owner, nonemployee). Imaging test overuse beliefs were associated with lower patient volume and location in an MSA < 1 million in size; PCPs who saw ≥26 breast cancer patients/year had lower odds of overuse beliefs for CXR, while PCPs practicing in MSAs ≥ 1 million had lower odds of overuse beliefs for ADI. Sensitivity analyses (N=489) excluding physicians reporting no involvement in surveillance testing yielded very similar results, except that blood test overuse beliefs were not associated with practice size while showing new associations with self-efficacy (p=.04) and defensive medicine (p=.05) (Supplemental Digital Content 1), and imaging test overuse beliefs showed new associations (p=.05) with perceived ambiguity about surveillance test recommendations (Supplemental Digital Content 2). The meaning of these new associations, however, is unclear since the between-level contrasts for these variables showed no clear trends or patterns, and most were not statistically significant. Any moderating effect of PCPs’ involvement in surveillance testing on the associations between overuse beliefs and the factors examined in this study thus appears small.
Table 2.
PCP (N=822) | Blood Tests | Imaging Tests | |||||
---|---|---|---|---|---|---|---|
All Tests vs. None | 1–2 Tests vs. None | p | CXR vs. None | ADI±CXR vs. None | p | ||
Practice Characteristics | |||||||
Practice size | 1 | 1.0 | 1.0 | .02 | |||
2–5 | 0.42 (0.21, 0.85) | 0.64 (0.32, 1.28) | |||||
≥ 6 | 0.36 (0.17, 0.73) | 0.46 (0.24, 0.90) | |||||
Breast cancer patient volume | 0–5/year | 1.0 | 1.0 | .04 | |||
6–15/year | 1.00 (0.63, 1.60) | 1.03 (0.54, 1.97) | |||||
16–25/year | 0.55 (0.30, 1.01) | 0.95 (0.53, 1.69) | |||||
≥ 26/year | 0.48 (0.29, 0.80) | 0.78 (0.44, 1.39) | |||||
Employment arrangement | Full/Part owner | 1.0 | 1.0 | .04 | |||
Employee physician owned | 0.85 (0.35, 2.09) | 1.60 (0.72, 3.58) | |||||
Employee-large group, HMO | 0.37 (0.18, 0.78) | 0.55 (0.29, 1.05) | |||||
Employee-university practice | 0.39 (0.11, 1.46) | 0.68 (0.23, 1.98) | |||||
Other | 0.46 (0.22, 0.94) | 0.39 (0.20, 0.80) | |||||
MSA | All other MSAs | 1.0 | 1.0 | <.001 | |||
1 million | 1.38 (0.92, 2.05) | 0.49 (0.34, 0.72) |
Multivariable analyses weighted to the US physician population and adjusted for age, race, sex, income, country of training, EMR use, proportion of uninsured patients, teaching hospital affiliation, self efficacy, perceived ambiguity, defensive medicine, use of survivorship care plans, involvement in cancer surveillance testing, and role communication—none of which showed significant associations with overuse beliefs
CXR = Chest X-ray; ADI = Advanced Diagnostic Imaging (bone scan, computed tomography (CT) scan, magnetic resonance imaging (MRI) scan); MSA = Metropolitan Statistical Area
Data source: Survey of Physician Attitudes Regarding the Care of Cancer Survivors (SPARCCS)
In contrast, more factors were significantly associated with oncologists’ test overuse beliefs (Table 3). Older age and international medical graduate status were associated with greater overuse beliefs for both blood tests and ADI, and lower self-efficacy and higher perceived ambiguity were associated with greater overuse beliefs for both blood tests and CXR. Other factors showed significant associations with overuse beliefs for one test type or the other. Race and employment arrangement were associated with blood test overuse beliefs; overuse was higher among Asian than white physicians and lower among employed physicians than practice owners. Patient volume was associated with overuse beliefs for imaging tests, although the pattern of association was non-linear and difficult to interpret.
Table 3.
Oncologists (N=922) | Blood Tests | Imaging Tests | |||||
---|---|---|---|---|---|---|---|
All Tests vs. None | 1–2 Tests vs. None | p | CXR vs. None | ADI±CXR vs. None | p | ||
Physician Characteristics | |||||||
Age | < 40 | 1.0 | 1.0 | .006 | 1.0 | 1.0 | <.001 |
40–49 | 0.95 (0.56, 1.61) | 1.17 (0.74, 1.85 | 0.71 (0.32, 1.58) | 1.14 (0.65, 2.03) | |||
50–59 | 1.38 (0.76, 2.51) | 1.31 (0.79, 2.16) | 0.58 (0.23, 1.47) | 1.98 (1.11, 3.56) | |||
≥ 60 | 4.01 (1.82, 8.85) | 2.19 (1.17, 4.13) | 1.47 (0.57, 3.81) | 3.81 (2.02, 7.16) | |||
Race | White | 1.0 | 1.0 | .04 | |||
Asian | 1.73 (0.98, 3.05) | 2.02 (1.26, 3.25) | |||||
Other | 0.67 (0.31, 1.46) | 1.03 (0.50, 2.11) | |||||
US Trained | Yes | 1.0 | 1.0 | .01 | 1.0 | 1.0 | .04 |
No | 1.96 (1.17, 3.29) | 0.91 (0.58, 1.43) | 1.44 (0.71, 2.91) | 1.74 (1.10, 2.76) | |||
Practice Characteristics | |||||||
Breast cancer patient volume | 0–10/week | 1.0 | 1.0 | .01 | |||
11–20/week | 0.33 (0.16, 0.70) | 0.86 (0.53, 1.40) | |||||
21–30/week | 0.50 (0.25, 1.02) | 1.10 (0.64, 1.91) | |||||
31+/week | 1.13 (0.51, 2.47) | 1.66 (0.94, 2.94) | |||||
Employment arrangement | Full/Part owner | 1.0 | 1,0 | .004 | |||
Employee - physician owned | 0.58 (0.26, 1.33) | 1.21 (0.62, 2.39) | |||||
Employee -large group, HMO | 0.58 (0.27, 1.26) | 0.49 (0.26, 0.93) | |||||
Employee - university practice | 0.51 (0.25, 1.05) | 0.41 (0.25, 0.69) | |||||
Other | 0.45 (0.21, 0.97) | 0.63 (0.32, 1.23) | |||||
Physician Attitudes and Practices | |||||||
Self-efficacy | Not at all/Somewhat | 1.0 | 1.0 | .03 | 1.0 | 1.0 | .01 |
Very confident | 0.41 (0.22, 0.79) | 0.62 (0.36, 1.08) | 0.38 (0.20, 0.73) | 1.07 (0.62, 1.83) | |||
Perceived ambiguity | Disagree | 1.0 | 1.0 | <..001 | 1.0 | 1.0 | .01 |
Don't know | 6.86 (1.44, 32.7) | 1.66 (0.38, 7.21) | 3.09 (0.32, 29.8) | 0.36 (0.07, 2.00) | |||
Agree | 2.78 (1.85, 4.16) | 1.69 (1.21, 2.37) | 3.00 (1.58, 5.70) | 1.34 (0.89, 2.01) |
Multivariable analyses weighted to the US physician population and adjusted for sex, income, practice size, MSA, EMR use, proportion of uninsured patients, teaching hospital affiliation, defensive medicine, use of survivorship care plans, and role communication—none of which showed significant associations with overuse beliefs.
CXR = Chest X-ray; ADI = Advanced Diagnostic Imaging (bone scan, computed tomography (CT) scan, magnetic resonance imaging (MRI) scan)
Data source: Survey of Physician Attitudes Regarding the Care of Cancer Survivors (SPARCCS)
Discussion
This study provides new evidence on US physicians’ beliefs about appropriate cancer surveillance testing in breast cancer survivors. To our knowledge, it is the first vignette-based study to examine this issue among both oncology specialists and PCPs, using a nationally representative sample. The current study thus extends past research, while offering new insights on the extent and potential determinants of physicians’ beliefs in overuse of an acknowledged low-value health care service.
A significant majority of PCPs and oncologists endorsed beliefs consistent with overuse of blood tests, while half of PCPs and 1/4 of oncologists endorsed overuse beliefs regarding imaging tests. The higher prevalence of overuse beliefs among PCPs than oncologists coincided with lower self-reported confidence in knowledge of breast cancer surveillance testing; 58% of PCPs vs. 15% of oncologists reported that they were “ not at all” or “ somewhat confident,” and 13% of PCPs vs. 2% of oncologists responded “ don’t know” to the item measuring perceived ambiguity about surveillance recommendations.
However, these cognitive factors—self-efficacy and perceived ambiguity—were associated with overuse beliefs only for oncologists, for whom they were among the strongest predictors of all variables. This suggests a critical influence of physician uncertainty on oncologists’ overuse of cancer surveillance testing. Physician uncertainty is thought to be a major cause of overuse in health care, and potential sources include physicians’ own lack of knowledge or experience, scientific uncertainty about the net benefits of health services, and the difficulty of determining outcomes for individuals due to random variation.38,39,50
Our study provides new information on the uncertainties specific to oncologists’ potential overuse of breast cancer surveillance tests and possible ways to reduce it. The association of overuse beliefs with low self-efficacy suggests that physicians’ lack of confidence in their knowledge of appropriate surveillance testing may prompt greater test ordering, and educational interventions to increase confidence and knowledge may reduce this propensity. Likewise, the association with perceived ambiguity about practice recommendations suggests that physicians respond to scientific uncertainty not by avoiding medical interventions—reflecting the phenomenon of “ ambiguity aversion”— but by initiating them.51 The inconsistency of this finding with data from patients and laypersons49,52 may reflect motivational factors unique to physicians—e.g., a medical culture that encourages thoroughness over efficiency,13,53 fear of malpractice litigation.43 These are factors that may moderate the effects of ambiguity on physicians’ decisions about the use of medical tests. Importantly, the results also suggest opportunities for remediation since ambiguity about breast cancer surveillance is arguably more perceived than real; current clinical practice guidelines agree in recommending mammography alone.1,2 Educational interventions that lessen perceived guideline ambiguity might thus reduce overuse, although the effectiveness of such an approach remains to be seen.
Several physician characteristics—older age, Asian race, international medical training—were associated with overuse beliefs among oncologists. These same factors have been shown to be associated with PCPs’ valuation of aggressive vs. conservative guidelines for cancer screening.48 Although not modifiable themselves, they suggest the influence of other factors—e.g., physician training, prevailing cultural norms—that may be modifiable.
It is unclear why physician characteristics and attitudinal factors associated with oncologists’ overuse beliefs did not show similar associations for PCPs. This might reflect PCPs’ more restricted role in cancer survivor care under the traditional “ oncologist-centered” delivery model, which might reduce the influence of perceptions and attitudes on test-ordering. The observed associations between practice characteristics (practice size, patient volume, practice ownership) and PCPs’ overuse beliefs also require explanation, and suggest that PCPs’ test overuse is driven more by structural than cognitive factors. These might include physician workload, access to peer or clinical decision support, financial incentives, and other system-level factors not ascertained by SPARCCS. It remains for further research to identify these and other unmeasured variables and confounders and to develop theoretical models and frameworks to guide further analyses of the determinants of surveillance test overuse.
Unexpectedly, we also found that neither use of cancer survivorship care plans nor role communication between PCPs and oncologists were associated with overuse beliefs. Although these practices have been promoted as strategies for making cancer survivor care more guideline-consistent, coordinated, and efficient,17 our data suggest they have limited influence although their future impact remains to be seen.28 We also found no association between overuse beliefs and defensive medical practice, although this factor may be difficult to accurately ascertain through self-report.
Our study had several limitations. SPARCCS only measured physicians’ beliefs, not actual practices, and relied solely on self-report which is susceptible to social desirability bias and recall error. To assess overuse beliefs, we used a single clinical vignette describing only one potential surveillance testing scenario, which requires validation. Furthermore, although responses to clinical vignettes have been shown to correlate well with actual physician practice,24–26 their validity in predicting cancer surveillance test overuse remains to be shown. Finally, the study’s cross-sectional nature limits causal inferences.
Despite these limitations, our study provides important evidence about the extent to which PCPs’ and oncologists’ beliefs about breast cancer surveillance reflect test overuse, and the factors associated with these beliefs. Physician uncertainty about appropriate surveillance testing may be an important and potentially remediable determinant of overuse beliefs among oncologists. Further work is needed to validate our findings by measuring actual overuse of cancer surveillance testing and its contributing factors, to test hypotheses about the influence of these factors, to reduce both perceived and real ambiguity in surveillance testing guidelines, and to devise interventions to help physicians and patients choose wisely and reduce surveillance test overuse in cancer survivor care.
Supplementary Material
Acknowledgments
Funding for the Survey of Physicians’ Attitudes Regarding the Care of Cancer Survivors (SPARCCS) was provided by the National Cancer Institute (contract number HSN261200700068C) and the American Cancer Society through its intramural research funds. Dr. Han had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. We thank the staff of Westat, Inc., for assistance in survey planning, design, and data collection, and other members of the SPARCCS team, including Noreen Aziz, Lynne Harlan, Julia Rowland, Tenbroeck Smith, Michael Stefanek, and Gordon Willis, for guidance and assistance at various stages of the study. The views expressed in this paper do not necessarily represent those of the National Cancer Institute or the American Cancer Society.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
Paul K.J. Han, Center for Outcomes Research and Evaluation, Maine Medical Center, Portland, ME; Tufts University School of Medicine, Boston, MA.
Carrie N. Klabunde, Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD.
Anne-Michelle Noone, Surveillance Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD.
Craig C. Earle, Ontario Institute for Cancer Research, Toronto, ON, Canada.
John Z. Ayanian, Division of General Medicine, Brigham and Women’s Hospital; Department of Health Care Policy, Harvard Medical School, Boston, MA.
Patricia A. Ganz, UCLA Schools of Public Health and Medicine, Jonsson Comprehensive Cancer Center, University of California, Los Angeles, CA.
Katherine S. Virgo, Health Services Research Program, Intramural Research Department, American Cancer Society, Atlanta, GA, USA.
Arnold L. Potosky, Georgetown University Medical Center, Lombardi Comprehensive Cancer Center, Washington, D.C..
References
- 1.Khatcheressian JL, Wolff AC, Smith TJ, et al. American Society of Clinical Oncology 2006 update of the breast cancer follow-up and management guidelines in the adjuvant setting. J Clin Oncol. 2006 Nov 1;24(31):5091–5097. doi: 10.1200/JCO.2006.08.8575. [DOI] [PubMed] [Google Scholar]
- 2.National Comprehensive Cancer Network. NCCN Clinical Practice Guidelines in Oncology.™ Breast Cancer Guidelines, Version 2.20082008 [Google Scholar]
- 3.Keating NL, Landrum MB, Guadagnoli E, Winer EP, Ayanian JZ. Surveillance testing among survivors of early-stage breast cancer. J Clin Oncol. 2007 Mar 20;25(9):1074–1081. doi: 10.1200/JCO.2006.08.6876. [DOI] [PubMed] [Google Scholar]
- 4.Grunfeld E, Hodgson DC, Del Giudice ME, Moineddin R. Population-based longitudinal study of follow-up care for breast cancer survivors. J Onc Pract. 2010 Jul;6(4):174–181. doi: 10.1200/JOP.200009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Richert-Boe KE. Heterogeneity of cancer surveillance practices among medical oncologists in Washington and Oregon. Cancer. 1995 May 15;75(10):2605–2612. doi: 10.1002/1097-0142(19950515)75:10<2605::aid-cncr2820751031>3.0.co;2-#. [DOI] [PubMed] [Google Scholar]
- 6.Chassin MR, Galvin RW. The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality. JAMA. 1998 Sep 16;280(11):1000–1005. doi: 10.1001/jama.280.11.1000. [DOI] [PubMed] [Google Scholar]
- 7.Hendee WR, Becker GJ, Borgstede JP, et al. Addressing overutilization in medical imaging. Radiology. 2010 Oct;257(1):240–245. doi: 10.1148/radiol.10100063. [DOI] [PubMed] [Google Scholar]
- 8.Ulmer C, Bruno M, Burke S, editors. Future Directions for the National Healthcare Quality and Disparities Reports. Washington, DC: Institute of Medicine; 2010. [PubMed] [Google Scholar]
- 9.Qaseem A, Alguire P, Dallas P, et al. Appropriate Use of Screening and Diagnostic Tests to Foster High-Value, Cost-Conscious Care. Ann Intern Med. 2012 Jan 17;156(2):147–149. doi: 10.7326/0003-4819-156-2-201201170-00011. [DOI] [PubMed] [Google Scholar]
- 10.Laine C. High-value testing begins with a few simple questions. Ann Intern Med. 2012 Jan 17;156(2):162–163. doi: 10.7326/0003-4819-156-2-201201170-00016. [DOI] [PubMed] [Google Scholar]
- 11.Cassel CK, Guest JA. Choosing wisely: helping physicians and patients make smart decisions about their care. JAMA. 2012 May 2;307(17):1801–1802. doi: 10.1001/jama.2012.476. [DOI] [PubMed] [Google Scholar]
- 12.ABIM Foundation. Choosing Wisely. [Accessed May 8, 2012];2012 http://choosingwiselyorg/, 2012.
- 13.Emanuel EJ, Fuchs VR. The perfect storm of overutilization. JAMA. 2008 Jun 18;299(23):2789–2791. doi: 10.1001/jama.299.23.2789. [DOI] [PubMed] [Google Scholar]
- 14.Grady D, Redberg RF. Less is more: how less health care can result in better health. Arch Intern Med. 2010 May 10;170(9):749–750. doi: 10.1001/archinternmed.2010.90. [DOI] [PubMed] [Google Scholar]
- 15.Smith TJ, Hillner BE. Bending the cost curve in cancer care. N Engl J Med. 2011 May 26;364(21):2060–2065. doi: 10.1056/NEJMsb1013826. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Centers for Disease Control and Prevention (CDC) Cancer survivors—United States, 2007. MMWR. 2007;60(9):269–272. [PubMed] [Google Scholar]
- 17.Hewitt M, Greenfield S, Stovall E, editors. From cancer patient to cancer survivor: Lost in transition. Washington, DC: National Academies Press; 2005. Institute of Medicine, ed. [Google Scholar]
- 18.Deyo RA. Cascade effects of medical technology. Ann Rev Public Health. 2002;23:23–44. doi: 10.1146/annurev.publhealth.23.092101.134534. [DOI] [PubMed] [Google Scholar]
- 19.Mold JW, Stein HF. The cascade effect in the clinical care of patients. N Engl J Med. 1986 Feb 20;314(8):512–514. doi: 10.1056/NEJM198602203140809. [DOI] [PubMed] [Google Scholar]
- 20.Field TS, Doubeni C, Fox MP, et al. Under utilization of surveillance mammography among older breast cancer survivors. J Gen Intern Med. 2008 Feb;23(2):158–163. doi: 10.1007/s11606-007-0471-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Lucas FL, Sirovich BE, Gallagher PM, Siewers AE, Wennberg DE. Variation in cardiologists' propensity to test and treat: is it associated with regional variation in utilization? Circulation. 2010 May;3(3):253–260. doi: 10.1161/CIRCOUTCOMES.108.840009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Sirovich B, Gallagher PM, Wennberg DE, Fisher ES. Discretionary decision making by primary care physicians and the cost of U.S. Health care. Health Affairs. 2008 May-Jun;27(3):813–823. doi: 10.1377/hlthaff.27.3.813. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Sirovich BE, Gottlieb DJ, Welch HG, Fisher ES. Variation in the tendency of primary care physicians to intervene. Arch Intern Med. 2005 Oct 24;165(19):2252–2256. doi: 10.1001/archinte.165.19.2252. [DOI] [PubMed] [Google Scholar]
- 24.Dresselhaus TR, Peabody JW, Luck J, Bertenthal D. An evaluation of vignettes for predicting variation in the quality of preventive care. J Gen Intern Med. 2004 Oct;19(10):1013–1018. doi: 10.1007/s11606-004-0003-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000 Apr 5;283(13):1715–1722. doi: 10.1001/jama.283.13.1715. [DOI] [PubMed] [Google Scholar]
- 26.Peabody JW, Luck J, Glassman P, et al. Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Ann Intern Med. 2004 Nov 16;141(10):771–780. doi: 10.7326/0003-4819-141-10-200411160-00008. [DOI] [PubMed] [Google Scholar]
- 27.Johnson FE, Johnson MH, Clemente MF, Paniello RC, Virgo KS. Geographical variation in surveillance strategies after curative-intent surgery for upper aerodigestive tract cancer. Ann Surg Oncol. 2006 Aug;13(8):1063–1071. doi: 10.1245/ASO.2006.04.014. [DOI] [PubMed] [Google Scholar]
- 28.Potosky AL, Han PK, Rowland J, et al. Differences Between Primary Care Physicians' and Oncologists' Knowledge, Attitudes and Practices Regarding the Care of Cancer Survivors. J Gen Intern Med. 2011 Jul 22;26(12):1403–1410. doi: 10.1007/s11606-011-1808-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Cheung WY, Neville BA, Cameron DB, Cook EF, Earle CC. Comparisons of patient and physician expectations for cancer survivorship care. J Clin Oncol. 2009 May 20;27(15):2489–2495. doi: 10.1200/JCO.2008.20.3232. [DOI] [PubMed] [Google Scholar]
- 30.Del Giudice ME, Grunfeld E, Harvey BJ, Piliotis E, Verma S. Primary care physicians' views of routine follow-up care of cancer survivors. J Clin Oncol. 2009 Jul 10;27(20):3338–3345. doi: 10.1200/JCO.2008.20.4883. [DOI] [PubMed] [Google Scholar]
- 31.Klabunde CN, Frame PS, Meadow A, Jones E, Nadel M, Vernon SW. A national survey of primary care physicians' colorectal cancer screening recommendations and practices. Prev Med. 2003 Mar;36(3):352–362. doi: 10.1016/s0091-7435(02)00066-x. [DOI] [PubMed] [Google Scholar]
- 32.Klabunde CN, Lanier D, Nadel MR, McLeod C, Yuan G, Vernon SW. Colorectal cancer screening by primary care physicians: recommendations and practices, 2006–2007. Am J Prev Med. 2009 Jul;37(1):8–16. doi: 10.1016/j.amepre.2009.03.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Keating NL, Landrum MB, Rogers SO, Jr, et al. Physician factors associated with discussions about end-of-life care. Cancer. 2010 Feb 15;116(4):998–1006. doi: 10.1002/cncr.24761. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Nissen MJ, Beran MS, Lee MW, Mehta SR, Pine DA, Swenson KK. Views of primary care providers on follow-up care of cancer patients. Fam Med. 2007 Jul-Aug;39(7):477–482. [PubMed] [Google Scholar]
- 35.American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 5th ed. Lenexa, KS: American Association for Public Opinion Research; 2008. [Google Scholar]
- 36.Orszag PR. Congressional Budget Office, ed. Washington, DC: Congressional Budget Office; 2008. The Overuse, Underuse, and Misuse of Health Care. [Google Scholar]
- 37.Owens DK, Qaseem A, Chou R, Shekelle P. High-value, cost-conscious health care: concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med. 2011 Feb 1;154(3):174–180. doi: 10.7326/0003-4819-154-3-201102010-00007. [DOI] [PubMed] [Google Scholar]
- 38.Bentley TG, Effros RM, Palar K, Keeler EB. Waste in the U.S. Health care system: a conceptual framework. Milbank Quart. 2008 Dec;86(4):629–659. doi: 10.1111/j.1468-0009.2008.00537.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Fuchs VR. Eliminating "waste" in health care. JAMA. 2009 Dec 9;302(22):2481–2482. doi: 10.1001/jama.2009.1821. [DOI] [PubMed] [Google Scholar]
- 40.Wennberg JE. Unwarranted variations in healthcare delivery: implications for academic medical centres. BMJ. 2002 Oct 26;325(7370):961–964. doi: 10.1136/bmj.325.7370.961. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Wennberg JE, Fisher ES, Skinner JS. Geography and the debate over Medicare reform. Health Affairs. 2002 Jul-Dec;:W96–W114. doi: 10.1377/hlthaff.w2.96. Suppl Web Exclusives. [DOI] [PubMed] [Google Scholar]
- 42.Donohoe MT. Comparing generalist and specialty care: discrepancies, deficiencies, and excesses. Arch Intern Med. 1998 Aug 10–24;158(15):1596–1608. doi: 10.1001/archinte.158.15.1596. [DOI] [PubMed] [Google Scholar]
- 43.Studdert DM, Mello MM, Sage WM, et al. Defensive medicine among high-risk specialist physicians in a volatile malpractice environment. JAMA. 2005 Jun 1;293(21):2609–2617. doi: 10.1001/jama.293.21.2609. [DOI] [PubMed] [Google Scholar]
- 44.Kerfoot BP, Holmberg EF, Lawler EV, Krupat E, Conlin PR. Practitioner-level determinants of inappropriate prostate-specific antigen screening. Arch Intern Med. 2007 Jul 9;167(13):1367–1372. doi: 10.1001/archinte.167.13.1367. [DOI] [PubMed] [Google Scholar]
- 45.Espeland A, Baerheim A. Factors affecting general practitioners' decisions about plain radiography for back pain: implications for classification of guideline barriers--a qualitative study. BMC Health Serv Res. 2003 Mar 24;3(1):8. doi: 10.1186/1472-6963-3-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Lysdahl KB, Hofmann BM. What causes increasing and unnecessary use of radiological investigations? A survey of radiologists' perceptions. BMC Health Serv Res. 2009;9:155. doi: 10.1186/1472-6963-9-155. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.van der Weijden T, van Bokhoven MA, Dinant GJ, van Hasselt CM, Grol RP. Understanding laboratory testing in diagnostic uncertainty: a qualitative study in general practice. Brit J Gen Pract. 2002 Dec;52(485):974–980. [PMC free article] [PubMed] [Google Scholar]
- 48.Han PK, Klabunde CN, Breen N, et al. Multiple clinical practice guidelines for breast and cervical cancer screening: perceptions of US primary care physicians. Med Care. 2011 Feb;49(2):139–148. doi: 10.1097/MLR.0b013e318202858e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Han PK, Kobrin SC, Klein WM, Davis WW, Stefanek M, Taplin SH. Perceived ambiguity about screening mammography recommendations: association with future mammography uptake and perceptions. Cancer Epidemiol Biomarkers Prev. 2007 Mar;16(3):458–466. doi: 10.1158/1055-9965.EPI-06-0533. [DOI] [PubMed] [Google Scholar]
- 50.Han PK, Klein WM, Arora NK. Varieties of uncertainty in health care: a conceptual taxonomy. Med Decis Making. 2011 Nov;31(6):828–838. doi: 10.1177/0272989X11393976. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Ayanian JZ, Berwick DM. Do physicians have a bias toward action? A classic study revisited. Med Decis Making. 1991 Jul-Sep;11(3):154–158. doi: 10.1177/0272989X9101100302. [DOI] [PubMed] [Google Scholar]
- 52.Camerer C, Weber M. Recent developments in modeling preferences: uncertainty and ambiguity. J Risk Uncertainty. 1992;5:325–370. [Google Scholar]
- 53.Fox RC. The evolution of medical uncertainty. Milbank Mem Fund Q Health Soc. 1980 Winter;58(1):1–49. [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.