Abstract
Background
Second medical opinions have become commonplace and even mandatory in some health-care systems, as variations in diagnosis, treatment or prognosis may emerge among physicians.
Objective
To evaluate whether physicians’ judgment is affected by another medical opinion given to a patient.
Design
Orthopedic surgeons and neurologists filled out questionnaires presenting eight hypothetical clinical scenarios with suggested treatments. One group of physicians (in each specialty) was told what the other physician’s opinion was (study group), and the other group was not told what it was (control group).
Participants
A convenience sample of 332 physicians in Israel: 172 orthopedic surgeons (45.9% of their population) and 160 neurologists (64.0% of their population).
Measurements
Scoring was by choice of less or more interventional treatment in the scenarios. We used χ2 tests and repeated measures ANOVA to compare these scores between the two groups. We also fitted a cumulative ordinal regression to account for the dependence within each physician’s responses.
Results
Orthopedic surgeons in the study group chose a more interventionist treatment when the other physician suggested an intervention than those in the control group [F (1, 170) = 4.6, p = 0.03; OR = 1.437, 95% CI 1.115-1.852]. Evaluating this effect separately in each scenario showed that in four out of the eight scenarios, they chose a more interventional treatment when the other physician suggested an intervention (scenario 1, p = 0.039; scenario 2, p < 0.001; scenario 3, p = 0.033; scenario 6, p < 0.001). These effects were insignificant among the neurologists [F (1,158) = 0.44, p = 0.51; OR = 1.087, 95% CI 0.811-1.458]. In both specialties there were no differences in responses by level of clinical experience [orthopedic surgeons: F (2, 166) = 0.752, p = 0.473; neurologists: F (2,154) = 1.951, p = 0.146].
Conclusions
The exploratory survey showed that in some cases physicians’ judgments may be affected by other physicians’ opinions, but unaffected in other cases. Weighing previous opinions may yield a more informed clinical decision, yet physicians may be unintentionally influenced by previous opinions. Second opinion has the potential to improve the clinical decision-making processes, and mechanisms are needed to reconcile discrepant opinions.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-012-2056-y) contains supplementary material, which is available to authorized users.
KEY WORDS: second-opinion, differential diagnosis, diagnostic reasoning, medical decision-making, health policy, surgery, orthopedics, neurology, surveys, consultation
INTRODUCTION
Second opinion is a treatment ratification tool with a critical influence on diagnosis, treatment and prognosis. Common sense indicates that ‘two pairs of eyes are better than one.’ Back in the 1970s, Massachusetts required that Medicaid recipients obtain a second surgical opinion before elective surgery.1 Indeed, vast research has identified significant diagnostic discrepancies between independent opinions in many clinical domains,2–12 and utilization review programs that require second opinions substantially reduce the number of diagnostic and surgical procedures.13,14 In recent years, second opinion has evolved into a patient's right,15 and many patients are likely to obtain a second opinion on a serious diagnosis.16 Second opinion has also become an integral part of many health-care systems, offering a competitive marketing benefit for attracting patients.
Research on second opinions has mainly focused on the prevalence of diagnostic variations between independent reviewers.2–12 Other studies dealt with the cost-effectiveness of second opinion programs,17–20 the patient’s reasons for seeking a second opinion and the characteristics of these patients,15,21–24 and patient-physician communication in second opinions.25–28 The other side of the coin—physician judgment in forming an opinion—has been scarcely studied to the best of our knowledge. Physicians giving second opinions may be affected by previous opinions given to the patient by other physicians. For example, in a recent survey, nearly two-thirds of 65 Australian oncologists believed that the first physician’s recommendation influenced the outcome of the second opinion.29 If such effects occur, they may have important implications on patient care. We sought to evaluate whether such effects empirically exist in second opinion consultations.
METHODS
Study Tool
We developed a questionnaire with eight hypothetical clinical scenarios representing clinical dilemmas encountered in routine practice, followed by closed-choice diagnostic and treatment options with no clear-cut “correct” choice (see online appendix). The scenarios aimed to cover a variety of clinical scenarios, in terms of clinical condition and age of patient. The options ranged from the least interventional (e.g., physiotherapy) to the most interventional (e.g., surgery). We asked the physicians to choose the most appropriate option. We used closed-choice questions because we aimed to study the influence of other opinions rather than testing clinical correctness. The physicians were asked to anonymously fill out their clinical experience, gender, and country of medical school at the end of the questionnaire. Clinical experience was categorized as: resident, senior with up to 7 years of post-training experience (2 years of fellowship and 5 years as an independent senior), and senior having more than 7 years of post-training experience. We had no a priori hypotheses about the influence of the demographic factors and basically used them as sample descriptors.
We developed separate (but similarly formatted) questionnaires for orthopedic surgeons and neurologists, each containing scenarios pertinent to each specialty. We chose specialties that involved high rates of second opinions, and enabled comparison between surgical and non-surgical specialties. The scenarios were prepared by an orthopedic surgeon (YK) and a neurologist (GI), and validated by senior peers. We performed a pilot study with 40 physicians to evaluate the clarity of the questionnaire and agreement with the suggested treatments. The pilot study showed that the scenarios were clear and answerable within a limited time of less than 10 min.
Study Design
The study design is presented in Figure 1. The first four scenarios appeared on the first page of the questionnaire, with no information about whether this was the first or second opinion (baseline scenarios). The other four scenarios appeared on the second page, under one of two conditions: one group of physicians was told what the other opinion was (study group), and the other group was told that the patient already got another opinion, but not what this opinion was (control group). Hence, each physician saw eight responses, consisting of four baseline scenarios and four ‘condition’ scenarios (either previous opinion is known or unknown). Such a design enables both a comparison between the two groups (conditions) and a comparison of each physician to his/her baseline. All physicians in each specialty viewed the same eight scenarios. To ensure that each of the eight scenarios appears both as a baseline scenario and as a context of second opinion, the four scenarios on the first page and the four scenarios on the second page were flipped across the two formats: in one format, scenarios 1-4 were on page 1 and scenarios 5-8 on page 2, and vice versa in the other format. Hence, we had four instruments for each specialty (2 conditions × 2 formats). Thus, each scenario appeared in a different context in the different instruments: as a baseline scenario, as an “other opinion known” scenario, or as an “other opinion unknown” scenario. To avoid possible confounding of the nature of the other opinion (e.g., more interventional or less interventional), the other opinion was always the most interventional one.
Study Population and Sampling Method
We administered the questionnaires to a convenience sample of orthopedic surgeons and neurologists. The questionnaires were administered during annual meetings of the Israeli Orthopedic Association and the Israeli Neurological Association, and in orthopedic and neurological wards at large teaching academic medical centers during 2008-2009. The questionnaire was interviewer-administered to ensure that baseline scenarios (1st page) were responded to prior to the ‘condition’ scenarios (2nd page). The participants were not provided with any incentive. The protocol was approved by the institutional ethics and human subjects review committees, and by the Israeli Ministry of Health, Department of Clinical Trials, for the nationwide surveys.
Outcome Measures
The outcome measure was an "interventional score" that represents the level of intervention chosen (i.e., choosing a more interventional treatment vs a less interventional one). The responses to each scenario were coded on a pre-defined ordinal scale from least interventional to most interventional (see online appendix). We summarized the "interventional score" at the physician level and at the scenario level. At the physician level, we summed each physician's responses across all scenarios, in the four baseline scenarios (1st page), and in the four condition scenarios (2nd page). At the scenario level, we summed the response scores to each scenario, across all physicians. Hence, each scenario had three sums: as a baseline scenario, as an “other opinion known” scenario, and as an “other opinion unknown” scenario.
Statistical Analyses
We analyzed the interventional scores at the physician level and at the scenario level. At the physician level, we used repeated measures ANOVA to compare between each physician’s sum of interventional scores in the baseline scenarios (1st page scenarios) and their sum of interventional scores in the ‘condition’ scenarios (2nd page scenarios) (the within-subjects variable), by the condition (first opinion is known or unknown), and clinical experience (the between-subjects variables). To adjust for the dependence within each physician’s responses, we fitted a cumulative ordinal regression, within the framework of generalized estimating equations.30 The dependent variable was the physician’s response for each scenario. The predictors were the context under which the physician saw the scenario (baseline, first opinion is known or unknown), the clinical experience, and their interaction as fixed factors. Because the scenarios had a different number of options (2, 3, or 4), we normalized the responses into three ordinal interventional levels: low, medium, and high. We used the physician’s identifier as a random effect for clustering. We used backward elimination for model selection. At the scenario level, we used the χ2 test for independence between the response level (1-4) and the context under which the scenario appeared (baseline, other opinion known, or unknown). We ran all statistical tests using SPSS® version 18 and ran them separately for each specialty.
RESULTS
A total of 332 questionnaires were collected, of which 172 were from orthopedic surgeons (45.9% of the 375 registered orthopedic surgeons in Israel) and 160 were from neurologists (64.0% of the 250 registered neurologists in Israel). There were significant differences in the characteristics of the two specialties (Table 1). The orthopedic surgeons’ sample included more male physicians [χ2(1) = 43.2, p < 0.001], fewer senior physicians [χ2(2) = 30.0, p < 0.001], and more physicians who studied in Israel [χ2(1) = 5.9, p < 0.015] compared to the neurologists’ sample.
Table 1.
Characteristics | Orthopedic surgeons (N = 172) % | Neurologists (N = 160) % | p |
---|---|---|---|
Gender (% male) | 96.4 | 65.6 | <0.001 |
Clinical experience | <0.001 | ||
Resident | 39.0 | 26.3 | |
Senior ≤7 years | 18.0 | 3.8 | |
Senior >7 years | 43.0 | 70.0 | |
Medical education country (% Israel) | 60.7 | 45.6 | 0.015 |
Responses to the Baseline Scenarios
The distribution of the responses to the baselines scenarios (see online appendix) shows that there was an adequate diversity of opinion, i.e., none of the scenarios had a dominant response across the physicians. The orthopedic surgeons tended to choose a less interventional treatment in scenarios 1, 2, 6, and 7, and a more interventional treatment in scenarios 4 and 5, while in scenarios 3 and 8 there was a balanced choice. The neurologists tended to choose a less interventional treatment in scenario 1, and a more interventional treatment in scenarios 5 and 8, while in scenarios 2, 3, 4, 6, and 7 there was a balanced choice. Hence, the scenarios described a range of various clinical scenarios with no clear-cut choice, with some tendency to fewer interventional treatments, thereby allowing an evaluation of how a more interventional choice might affect a physician’s judgment.
Interventional Scores at the Physician Level
In the orthopedic surgeons’ sample, physicians assigned to both the study group and the control group had similar mean scores for the four baseline scenarios (1.85 vs 1.95; p = 0.45). Yet in the second set of four scenarios (the ‘condition’ scenarios), study group physicians had significantly higher mean interventional scores than the control group physicians (2.25 vs 1.97; p = 0.03). For the study group physicians, the mean interventional scores were significantly higher in the ‘condition’ scenarios than their baseline interventional scores (2.25 vs 1.85; p < 0.01), whereas for the control group physicians, these mean interventional scores were not different from their baseline interventional scores (1.97 vs 1.95; p = 0.87) (Fig. 2a). In the neurologists’ sample, physicians assigned to both the study group and the control group had similar mean scores for the four baseline scenarios (1.93 vs 1.92; p = 0.97) and the second set of four scenarios (the ‘condition’ scenarios) (2.00 vs 1.91; p = 0.51). For both the study and control groups, these mean interventional scores were not different from their baseline interventional scores (study group: 2.00 vs 1.93; p = 0.64; control group: 1.91 vs 1.92; p = 0.89) (Fig. 2b). In both specialties there were no differences in responses by level of training [orthopedic surgeons: F (2, 166) = 0.752, p = 0.47; neurologists: F (2,154) = 1.951, p = 0.15].
The cumulative ordinal regression models supported these findings. In the orthopedic sample, the probability to choose a more interventional treatment was greater in the study group relative to the baseline (OR = 1.437, 95% CI 1.115-1.852), while in the control group it was not different from the baseline (OR = 1.059, 95% CI 0.871-1.286). In the neurologists’ sample, the probability to choose a more interventional treatment was not different from the baseline in both the study group (OR = 1.087, 95% CI 0.811-1.458) and the control group (OR = 1.017, 95% CI 0.827-1.250).
In simple words, the orthopedic surgeons who knew that the other physician had suggested an intervention (study group), chose a more interventional treatment compared to the baseline and to those who did not know that the other physician suggested an intervention (control group). The neurologists were not affected by whether they knew the previous opinion or not.
Interventional Scores at the Scenario Level
Estimating these effects separately per each scenario showed that in four out of the eight scenarios, the orthopedic surgeons chose a more interventional treatment when they knew that the other physician suggested an intervention (scenario 1, p = 0.036; scenario 2, p < 0.001; scenario 3, p = 0.015; scenario 6, p < 0.001) (Fig. 3a). For these scenarios we performed post-hoc multiple comparisons among the baseline, the study group and the control group with Holm’s correction,31 which yields adjusted p-values. Significant differences were found between the baseline and the study group (scenario 1, p = 0.039; scenario 2, p < 0.001; scenario 3, p = 0.033; scenario 6, p < 0.001). The interventional score in these scenarios was greater in the study group relative to the baseline. Again, no such effects were found among the neurologists (Fig. 3b).
DISCUSSION
Our findings show that awareness of another opinion shifted orthopedic surgeons’ choices towards a previously given more interventional opinion. These findings empirically support a recent survey in which nearly two-thirds of 65 Australian oncologists believed that the first physician’s recommendation influenced the outcome of the second opinion.29 However, the orthopedic surgeons’ judgment shifted in specific scenarios, but was not shifted in other scenarios. The neurologists were unaffected by previously given opinions.
Our study shows empirically that a previous medical opinion might influence medical decision-making in specific physician populations and specific clinical scenarios. We are cautious about making a judgment whether being affected by a previous opinion is “good” or “bad,” or about asserting its generalizability to other populations. Nevertheless, our results should generate hypotheses and highlight areas for further research rather than making a statement about general physician behavior.
What is the Clinical Meaning of These Findings?
It is reasonable to expect multiple solutions to clinical problems. Indeed, surgical judgment can differ radically from one surgeon to another.32 Interventional treatments such as surgery sometimes eventually appear to be unnecessary,33 and might have subsequent complications and complex rehabilitation. Conversely, delaying necessary surgery might have deleterious effects, to the point of requiring a more radical intervention than the one that would have sufficed beforehand. Two similar opinions would make a strong argument, but discrepant opinions may provide the impetus for deeper evaluation and reconciliation of the discrepancies. This is especially relevant considering that patients tend to give the second opinion greater importance than the first one.34 Moreover, information from previous opinions may save time and costs of redundant diagnostic procedures, but awareness of previous opinions may influence the consultant’s judgment, making it seemingly less objective.
Why Were Orthopedic Surgeons Affected and Neurologists Not?
One may wonder how differences between orthopedic surgeons and neurologists might evolve in the effect of the previous opinions. First, we assume that the nature and characteristics of each specialty may contribute to different clinical judgments and attitudes towards other opinions. Second, differences in the treatment options available for the clinical scenarios may also have played a role: the orthopedic scenarios had a wider treatment spectrum (i.e., the range between the least interventional to most interventional treatment, e.g., physiotherapy vs arthroscopic capsular shift), whereas the neurological scenarios had a less wide treatment and diagnostic spectra (e.g., begin an immune-modulatory treatment vs expectant clinical follow-up). Third, the two samples had different characteristics (gender, clinical experience, and country of training). However, we did not observe an effect of these characteristics on the physicians’ judgment. However, this study mainly intended to probe whether the discussed decision effects empirically exist, and further research is required to study the reasons for differences in the clinical rationale between orthopedics and neurology.
Other Factors that May Affect Judgment
Besides the previous opinion, other clinical, cultural, and organizational factors may affect clinical judgment and the effect of other opinions. However, seniority did not play a significant role in our data. Nevertheless, it may have had an effect, hypothetically, if the respondent had known what the other physician’s seniority was. Moreover, all of the participating physicians practice in academic hospitals where each department follows established protocols that are followed by both junior and senior physicians. Personality traits and economic considerations may also play a role in decision-making. Following another opinion also may be a safer option because it is less prone to future law suits.35,36 Future studies should explore such issues in other clinical, cultural, and organizational settings.
Strengths and Limitations
The strengths of this study are that it was the first to attempt to study a unique physician decision-making situation in a second opinion context, with a sample size covering a large proportion of the studied populations. The achieved response rates conform to physicians' response rates in published surveys, varying between 30-70%.29,37–39 The instrument we used was adequate to detect potential influences from second opinions, because it compares between each physician’s natural (baseline) responses and responses while being aware of another opinion. We acknowledge some limitations to this study. The questionnaires reflect attitudes towards hypothetical scenarios, with incomplete information about the patient and the condition. Yet physicians in real-life settings are frequently required to make decisions with imperfect information. The scenarios were intentionally presented without a clear-cut choice, because we did not look for a "right" clinical answer but for the effect of the other opinion. However, other choices may inevitably emerge in real-life settings. Second, in an instrument such as the one we used (comparing within-subject baseline to ‘condition’ responses), it may have been challenging for the physicians to imagine how they would respond in real life to a situation of decision-making considering another opinion, as compared to decision-making without another opinion. Third, the other opinion provided was always the most interventional one. We were unable to add additional experimental conditions because of the population saturation. Hence, it would be interesting to know whether such effects would have existed had the other opinion been the least interventional one or an ambivalent one.
CONCLUSIONS
This exploratory survey showed that physician judgment in some cases may be affected by other physicians’ opinions, but unaffected in other cases. The orthopedic surgeons were affected in some cases by a previous opinion that suggested an intervention, while the neurologists were not affected by whether they knew the previous opinion or not. Due to the impact of any decision on the patient's health, clinicians should be mindful of such potential effects. Further studies should explore such responses in other clinical, cultural, and organizational settings.
Electronic Supplementary Material
Acknowledgments
We thank the physicians who participated in the survey. We would like to thank David Shinar, PhD, for his help with the study concept and design. We thank Alex Aviram, MD, and Roni Gamzo, MD, PhD, for their comprehensive review of an earlier draft, and Simone Moran, PhD, Ran Balicer, MD, PhD, and Joachim Meyer, PhD, for helpful insights. Additionally, we thank Mrs. Esther Eshkol for editorial assistance.
Financial support
Funding was provided by a grant from the Israel National Institute for Health Policy and Health Service Research (NIHP R/07/94).
Conflict of Interest
The authors declare that they do not have a conflict of interest.
References
- 1.Gertman PM, Stackpole DA, Levenson DK, Manuel BM, Brennan RJ, Janko GM. Second opinions for elective surgery. The mandatory Medicaid program in Massachusetts. N Engl J Med. 1980;302(21):1169–74. doi: 10.1056/NEJM198005223022103. [DOI] [PubMed] [Google Scholar]
- 2.Althabe F, Belizan JM, Villar J, et al. Mandatory second opinion to reduce rates of unnecessary caesarean sections in Latin America: a cluster randomized controlled trial. Lancet. 2004;363(9425):1934–40. doi: 10.1016/S0140-6736(04)16406-4. [DOI] [PubMed] [Google Scholar]
- 3.Graboys TB, Biegelsen B, Lampert S, Blatt CM, Lown B. Results of a second-opinion trial among patients recommended for coronary angiography. JAMA. 1992;268(18):2537–40. doi: 10.1001/jama.1992.03490180069028. [DOI] [PubMed] [Google Scholar]
- 4.Han KS, Joung JY, Cho KS, et al. Results of repeated transurethral resection for a second opinion in patients referred for nonmuscle invasive bladder cancer: The referral cancer center experience and review of the literature. J Endourol. 2008;22(12):2699–704. doi: 10.1089/end.2008.0281. [DOI] [PubMed] [Google Scholar]
- 5.Kronz JD, Westra WH. The role of second opinion pathology in the management of lesions of the head and neck. Curr Opin Otolaryngol Head Neck Surg. 2005;13(2):81–4. doi: 10.1097/01.moo.0000156162.20789.66. [DOI] [PubMed] [Google Scholar]
- 6.Briggs GM, Flynn PA, Worthington M, Rennie I, McKinstry CS. The role of specialist neuroradiology second opinion reporting: Is there added value? Clin Radiol. 2008;63(7):791–5. doi: 10.1016/j.crad.2007.12.002. [DOI] [PubMed] [Google Scholar]
- 7.Tomaszewski JE, Bear HD, Connally JA, et al. Consensus conference on second opinions in diagnostic anatomic pathology. Who, what, and when. Am J Clin Pathol. 2000;114(3):329–35. doi: 10.1093/ajcp/114.3.329. [DOI] [PubMed] [Google Scholar]
- 8.McSherry CK, Chen PJ, Worner TM, Kupferstein N, McCarthy EG. Second surgical opinion programs: Dead or alive? J Am Coll Surg. 1997;185(5):451–6. [PubMed] [Google Scholar]
- 9.Vierhout WP, Knottnerus JA, Ooij A, et al. Effectiveness of joint consultation sessions of general practitioners and orthopedic surgeons for locomotor-system disorders. Lancet. 1995;346(8981):990–4. doi: 10.1016/S0140-6736(95)91686-5. [DOI] [PubMed] [Google Scholar]
- 10.Graboys TB, Headley A, Lown B, Lampert S, Blatt CM. Results of a second-opinion program for coronary artery bypass graft surgery. JAMA. 1987;258(12):1611–4. doi: 10.1001/jama.1987.03400120061025. [DOI] [PubMed] [Google Scholar]
- 11.Meyer JE, Eberlein TJ, Stomper PC, Sonnenfeld MR. Biopsy of occult breast lesions. analysis of 1261abnormalities. JAMA. 1990;263(17):2341–3. doi: 10.1001/jama.1990.03440170063037. [DOI] [PubMed] [Google Scholar]
- 12.Morrow M, Jagsi R, Alderman AK, et al. Surgeon recommendations and receipt of mastectomy for treatment of breast cancer. JAMA. 2009;302(14):1551–6. doi: 10.1001/jama.2009.1450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Rosenberg SN, Allen DR, Handte JS, et al. Effect of utilization review in a fee-for-service health insurance plan. N Engl J Med. 1995;333(20):1326–30. doi: 10.1056/NEJM199511163332006. [DOI] [PubMed] [Google Scholar]
- 14.Myers SA, Gleicher N. A successful program to lower cesarean-section rates. N Engl J Med. 1988;319(23):1511–6. doi: 10.1056/NEJM198812083192304. [DOI] [PubMed] [Google Scholar]
- 15.Wagner TH, Wagner LS. Who gets second opinions? Health Aff (Millwood) 1999;18(5):137–45. doi: 10.1377/hlthaff.18.5.137. [DOI] [PubMed] [Google Scholar]
- 16.1997 public opinion of patient safety issues. Available at: www.npsf.org/wp-content/uploads/2011/10/Public_Opinion_of_Patient_Safety_Issues.pdf. Accessed March 15, 2012.
- 17.Coblentz TR, Mills SE, Theodorescu D. Impact of second opinion pathology in the definitive management of patients with bladder carcinoma. Cancer. 2001;91(7):1284–90. doi: 10.1002/1097-0142(20010401)91:7<1284::AID-CNCR1130>3.0.CO;2-E. [DOI] [PubMed] [Google Scholar]
- 18.Epstein JI, Walsh PC, Sanfilippo F. Clinical and cost impact of second-opinion pathology. Review of prostate biopsies prior to radical prostatectomy. Am J Surg Pathol. 1996;20(7):851–7. doi: 10.1097/00000478-199607000-00008. [DOI] [PubMed] [Google Scholar]
- 19.McCarthy EG, Finkel ML, Ruchlin HS. Second opinions on elective surgery. The Cornell/New York Hospital Study. Lancet. 1981;1(8234):1352–4. doi: 10.1016/S0140-6736(81)92527-7. [DOI] [PubMed] [Google Scholar]
- 20.Thompson JN, Varley CK, McClellan J, et al. Second opinions improve ADHD prescribing in a Medicaid-insured community population. J Am Acad Child Adolesc Psychiatry. 2009;48(7):740–8. doi: 10.1097/CHI.0b013e3181a2b2ed. [DOI] [PubMed] [Google Scholar]
- 21.Moumjid N, Gafni A, Bremond A, Carrere MO. Seeking a second opinion: Do patients need a second opinion when practice guidelines exist? Health Policy. 2007;80(1):43–50. doi: 10.1016/j.healthpol.2006.02.009. [DOI] [PubMed] [Google Scholar]
- 22.Axon A, Hassan M, Niv Y, Beglinger C, Rokkas T. Ethical and legal implications in seeking and providing a second medical opinion. Dig Dis. 2008;26(1):11–7. doi: 10.1159/000109379. [DOI] [PubMed] [Google Scholar]
- 23.Sutherland LR, Verhoef MJ. Patients who seek a second opinion: Are they different from the typical referral? J Clin Gastroenterol. 1989;11(3):308–13. doi: 10.1097/00004836-198906000-00013. [DOI] [PubMed] [Google Scholar]
- 24.Elder NC, Jacobson CJ, Zink T, Hasse L. How experiencing preventable medical problems changed patients' interactions with primary health care. Ann Fam Med. 2005;3(6):537–44. doi: 10.1370/afm.346. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Goldman RE, Sullivan A, Back AL, Alexander SC, Matsuyama RK, Lee SJ. Patients' reflections on communication in the second-opinion hematology-oncology consultation. Patient Educ Couns. 2009;76(1):44–50. doi: 10.1016/j.pec.2008.11.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Oskay-Ozcelik G, Lehmacher W, Konsgen D, et al. Breast cancer patients' expectations in respect of the physician-patient relationship and treatment management results of a survey of 617 patients. Ann Oncol. 2007;18(3):479–84. doi: 10.1093/annonc/mdl456. [DOI] [PubMed] [Google Scholar]
- 27.Sato T, Takeichi M, Hara T, Koizumi S. Second opinion behavior among Japanese primary care patients. Br J Gen Pract. 1999;49(444):546–50. [PMC free article] [PubMed] [Google Scholar]
- 28.Dalen I, Groothoff J, Stewart R, Spreeuwenberg P, Groenewegen P, Horn J. Motives for seeking a second opinion in orthopaedic surgery. J Health Serv Res Policy. 2001;6(4):195–201. doi: 10.1258/1355819011927486. [DOI] [PubMed] [Google Scholar]
- 29.Philip J, Gold M, Schwarz M, Komesaroff P. An exploration of the dynamics and influences upon second medical opinion consultations in cancer care. Asia-Pacific Journal of Clinical Oncology. 2011;7(1):41–6. doi: 10.1111/j.1743-7563.2010.01330.x. [DOI] [PubMed] [Google Scholar]
- 30.McCulloch CE, Searle SR. Generalized, linear, and mixed models. New York: J. Wiley; 2001. pp. 208–12. [Google Scholar]
- 31.Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat. 1979;6:65–70. [Google Scholar]
- 32.Rutkow IM, Gittelsohn AM, Zuidema GD. Surgical decision making. the reliability of clinical judgment. Ann Surg. 1979;190(3):409–19. doi: 10.1097/00000658-197909000-00017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Leape LL. Unnecessary surgery. Health Serv Res. 1989;24(3):351–407. [PMC free article] [PubMed] [Google Scholar]
- 34.McCarthy EG, Finkel ML. Second consultant opinion for elective orthopedic surgery. Am J Public Health. 1981;71(11):1233–6. doi: 10.2105/AJPH.71.11.1233. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Danzon P. The frequency and severity of medical malpractice claims. J Law Econ. 1984;27(1):115–48. doi: 10.1086/467060. [DOI] [PubMed] [Google Scholar]
- 36.Neale G. Reducing risks in gastroenterological practice. Gut. 1998;42(1):139–42. doi: 10.1136/gut.42.1.139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Asch D, Jedrziewski M, Christakis N. Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997;50(10):1129–36. doi: 10.1016/S0895-4356(97)00126-1. [DOI] [PubMed] [Google Scholar]
- 38.Cummings S, Savitz L, Konrad T. Reported response rates to mailed physician questionnaires. Health Serv Res. 2001;35(6):1347–55. [PMC free article] [PubMed] [Google Scholar]
- 39.Jepson C, Asch D, Hershey J, Ubel P. In a mailed physician survey, questionnaire length had a threshold effect on response rate. J Clin Epidemiol. 2005;58(1):103–5. doi: 10.1016/j.jclinepi.2004.06.004. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.