Abstract
Objective
Patients consider many factors when deciding how to receive medical care. This study used best-worst scaling (BWS), a technique novel to otolaryngology, to quantitatively examine preferences among patients choosing a laryngologist. Our objective was to quantify in a pilot cohort the relative importance patients place on a variety of attributes when seeking a laryngologist.
Study Design
BWS survey.
Setting
Academic voice clinic.
Methods
New patients were recruited to take a computerized BWS survey developed using attributes derived from patient input, expert opinion, and literature review. Attributes were grouped into 4 categories: physician reputation, physician qualifications, hospital-related factors, and other nonclinical factors. Responses were analyzed using multinomial logit regression to determine importance scores and associations with other variables.
Results
Eighty-seven of 93 patients recruited participated (93.5% response rate). Physician qualifications were the most important attributes to patients, with specialty laryngology training receiving the highest importance score (20.8; 95% CI, 20.2 to 21.5; P < .0001). Recommendations from referring physicians (15.6; 95% CI, 14.3 to 16.9) and use of cutting-edge technology (11.9; 95% CI, 10.7 to 13.1) were the second and third most important, respectively. Least important were nonclinical factors, including wait time to get an appointment (4.3; 95% CI, 2.8 to 5.8) and convenience of office location (1.5; 95% CI, 0.9 to 2.1). Just over half of patients (51.2%) reported willingness to wait 4 weeks for an appointment with a laryngologist. Older patients were less concerned with convenience-related factors.
Conclusion
Nonclinical factors were less important to patients than clinical factors, and laryngology-specific training was paramount. Stated preference methodologies can elucidate underlying preferences and help providers make care more patient centered.
Keywords: best-worst scaling, laryngology, dysphonia, voice, otolaryngology, patient preference, stated preference
Many factors are relevant to patients when choosing where and how to receive medical care. These preferences may pertain to structural aspects of care (eg, wait time, cost, convenience) or to outcomes (eg, highly efficacious or low-risk treatments). In otolaryngology, many decisions that patients make are preference sensitive,1 but there is little existing information about how patients choose a laryngologist. Given that understanding patient preferences has been demonstrated to improve decision making,2-4 it is imperative to rigorously and reproducibly study how patients choose their providers.
Stated preference methodologies have become an accepted means of measuring preferences pertaining to medical goods and services.5 These include contingent valuation, discrete choice experimentation, and best-worst scaling (BWS). BWS is a method that asks participants to identify the best and worst options among several choices and, with iterative selections, develops rankings of the options. Results from BWS studies on patient preferences allow researchers to understand the relative importance of different components of choices or preferences in a way that traditional Likert scale data cannot.6
Using BWS, we aimed to understand the relative importance patients place on a variety of attributes when seeking a laryngologist. The indications for consultation are many: complaints related to voicing, swallowing, breathing, and globus are common. For example, 1 in 13 patients presenting to a primary care doctor’s office may have dysphonia,7 and the lifetime prevalence of globus is approximately 22%.8 Patients seeking subspecialty care may present independently or at the request of referring physicians, and they may seek a laryngologist with a specific location, credential, or technology. However, the preferences that underpin these decisions are not well understood. To our knowledge, this is the first study to apply BWS to understand why patients present for laryngology consultation.
Materials and Methods
Survey Attribute Development
Survey development began with a review of previous literature relevant to patient preferences regarding choice of hospitals and practitioners, given that no specific research existed regarding choice of a laryngologist.9-17 A broad list of potentially relevant treatment attributes was combined using these sources. These were organized using a previously published structure that included categories for physician, hospital, and nonclinical factors.18 Further investigation of these attributes was then undertaken using published guidelines in the field of stated preference research.19-22 A pilot group of 10 patients was purposively selected to represent the population of the laryngology clinic. This group engaged in semistructured interviews about what they considered the most important attributes associated with care delivery in the laryngology clinic. These were conducted by research staff, one-on-one. These patients were then shown the list of treatment attributes generated in the prior literature review and asked to comment. Comments as were scrutinized by 2 expert reviewers for themes. These themes were grouped into broad categories. Overlapping attributes were consolidated. After this, 13 attributes remained in the survey ( Table 1 ), based on (1) patient interviews, (2) expert opinion, and (3) literature review. This set was tested on 10 additional patients to ensure comprehension.
Table 1.
List of 13 Attributes Selected for Survey.
| Abbreviation | Category | Description |
|---|---|---|
| Physician | Physician reputation | Recommendation from another doctor |
| Family/friends | Physician reputation | Recommendation from friends/family |
| Online reviews | Physician reputation | Online patient reviews of doctor/surgeon |
| Experience | Physician qualifications | Number of years your doctor has been in practice |
| Training | Physician qualifications | Doctor’s specialized training in laryngology |
| Surgeon caseload | Physician qualifications | Number of surgeries your doctor has performed |
| Equipment | Hospital-related factors | Use of cutting-edge equipment for diagnosis and treatment |
| Ancillary services | Hospital-related factors | Voice/speech therapy availability at hospital |
| Hospital focus | Hospital-related factors | Specialty focus of hospital |
| Hospital reputation | Hospital-related factors | Hospital’s reputation in the community |
| Location | Nonclinical factors | Convenience of the office location |
| Wait time | Nonclinical factors | Wait time to get an appointment |
| Insurance | Nonclinical factors | Doctor is covered by your insurance |
Study approval was obtained from the Massachusetts Eye & Ear Institutional Review Board.
Survey Design
We developed a 3-part web-based survey. One component related to gathering demographic information. A second component asked multiple-choice questions about care delivery attributes, including how long the patient had waited for an appointment, willingness to wait for an appointment, distance traveled to the appointment, chief complaint, and a rating of how bothersome their complaint was. These questions acted as comparators for the BWS exercise to assess for reliability.
The final component consisted of a BWS exercise. We evaluated various stated preference elicitation formats and selected BWS because our goal was to assess prioritization of preferences in a simple format; BWS is a well-accepted, efficient, and easily understood type of state preference research.23,24 BWS asks patients to pick the “best” and “worst” option from a list of attributes and then repeat this for multiple different lists.6 The questions for this task are created a priori using a balanced incomplete block design. This design is generated to show each participant every attribute an equal number of times (“balanced”) over a fixed number (herein, 12) of questions. Not all attributes were included in every questions (“incomplete block”). One thousand potential designs were computed (Lighthouse Studio); the most balanced design was chosen. Question order was randomized for each participant.
An example of a choice question from this study is shown in Figure 1 . Each participant was asked to pick the “most important” and “least important” attribute from a selection of 5 attributes drawn from the 13-attribute list previously generated; this task would then be completed 11 more times with different attributes listed. In this way, each attribute could be methodically compared to other attributes in a best/worst format. Each selection gives several pieces of ordinal data. For example, if a participant selects attribute “A” as the best and attribute “E” as the worst, we know that the participant values A > B, A > C, A > D, and A > E, as well as B > E, C > E, and D > E. Thus, the participants’ preferences for attributes emerge through segmented best/worst choice tasks.
Figure 1.
Example of a best-worst scaling choice question.
Participants
A convenience sample of adults seeking care at an outpatient laryngology clinic at a specialty otolaryngology and ophthalmology hospital were eligible to participate. Patients presenting on Thursdays over a period of 6 months were asked to participate. Inclusion criteria included new patients presenting for laryngology-related complaints, including those related to voice, swallowing, airway concerns, globus, and benign and malignant growths. Patients excluded were those who were established patients within the practice; those without English proficiency, as the survey was given in English; those without the cognitive or motor capabilities to complete the computerized survey; children (<18 years old); and those who elected not to participate. All eligible patients were privately approached at the end of the initial clinical visit by a research assistant (V.F.). Those who agreed to participate then undertook the computerized survey at a workstation compliant with hospital privacy policies. Surveys were taken by the patient alone, but a research assistant was available throughout for any questions.
Quality Control and Statistical Analysis
Several measures of quality control were imposed on the BWS data. First, total time to complete the survey was recorded. Those who took less than 3 minutes to fill out the survey were excluded; this time threshold was judged by the researchers to be the fastest a person could accurately fill out this survey. Second, root likelihood (RLH) was calculated for each participant. This is a measure of how well a modeled solution fits data, with 1.0 being the best possible and the worst being 1/n, where n is the number of choices available in the task; herein n = 5, and 0.2 is the worst RLH. Those having an RLH less than 0.227 were excluded.25 This was determined by calculating importance scores for 300 randomly responding simulated participants and using the upper 95% RLH as the cutoff. This process allowed us to eliminate random responders with 95% confidence. Finally, patients who did not log answers for the BWS exercise were excluded.
Analysis of the data was performed using several programs. Descriptive statistics (percentages, means, and standard deviations) were performed using Microsoft Excel. Regressions, including multinomial logit modeling, were performed using JMP 14 Pro and Lighthouse Studio to determine importance scores and their associations with sex (using unpaired t tests) and age (using Pearson correlation). Importance scores suggest how much each attribute has upon choice, and they are calculated for each individual by rescaling the coefficients of the multinomial logit model so that the sum of all scores is 100. Thus, importance scores may range from 0 (least important) to 100 (most important) scale. These scores were then averaged across the population. Confidence intervals (at 95%) were calculated for each importance score at the group level.
Results
Ninety-three patients were approached, and 3 declined to participate. Of the 90 patients who agreed to participate, 3 were excluded based on incomplete responses to the BWS questionnaire; none were excluded on the basis of time to complete <3 minutes or RLH criteria. Demographic characteristics of the population are shown in Table 2 . Most (55%) respondents were female, and the mean age was 55. The sample was largely white (85.1%), college educated (70.1%), and married (62.1%). Approximately 12.5% of the population self-identified as singers. Most chief complaints related to voice (36.8%), although breathing, swallowing, and globus sensation were all prevalent. Approximately 54% of patients rated their complaint as “very” or “extremely” bothersome. Median completion time for the survey was 8.7 minutes (range, 4.1-24.7 minutes).
Table 2.
Basic and Demographic Respondent Data (N = 87).a
| Characteristic | Value |
|---|---|
| Age, mean (SD), y | 55.0 (15.6) |
| Sex | |
| Male | 37 (42.5) |
| Female | 47 (54.0) |
| Prefer not to answer | 2 (2.3) |
| Blank | 1 (1.1) |
| Race | |
| White | 74 (85.1) |
| Black | 3 (3.4) |
| American Indian or Alaska Native | 0 (0.0) |
| Asian | 2 (2.3) |
| Hawaiian or Pacific Islander | 0 (0.0) |
| Prefer not to answer | 7 (8.0) |
| Blank | 1 (1.1) |
| Hispanic | |
| Yes | 3 (3.4) |
| No | 80 (92.0) |
| Prefer not to answer | 3 (3.4) |
| Blank | 1 (1.1) |
| Education | |
| Grade school | 2 (2.3) |
| High school | 22 (25.3) |
| College | 36 (41.4) |
| Advanced degree | 25 (28.7) |
| Prefer not to answer | 1 (1.1) |
| Blank | 1 (1.1) |
| Marital status | |
| Single, never married | 18 (20.7) |
| Married or domestic partnership | 54 (62.1) |
| Divorced or separated | 10 (11.5) |
| Widowed | 2 (2.3) |
| Prefer not to answer | 2 (2.3) |
| Blank | 1 (1.1) |
| Singer | |
| No | 75 (86.2) |
| Yes—nonprofessional | 10 (11.4) |
| Yes—professional | 1 (1.1) |
| Blank | 1 (1.1) |
| Chief complaint | |
| Voice | 32 (36.8) |
| Swallowing | 12 (13.8) |
| Breathing | 8 (9.2) |
| Feeling lump/tickle in throat | 7 (8.0) |
| Sore throat | 7 (8.0) |
| Cough | 6 (6.9) |
| Cancer | 2 (2.3) |
| Noncancerous growth | 1 (1.1) |
| Other | 12 (13.8) |
Values are presented as number (%) unless otherwise indicated.
The importance of each attribute is shown in Figure 2 , with 95% confidence intervals indicating the precision around each estimated importance score. A doctor with specialized training in laryngology was deemed the most important factor in choice of a laryngologist, with the highest importance score (20.8; 95% CI, 20.2-21.5; P < .0001). Recommendations from referring physicians (15.6; 95% CI, 14.3-16.9) and use of cutting-edge technology (11.9; 95% CI, 10.7-13.1) were the second and third most important, respectively. Less important were nonclinical factors, including wait time to get an appointment (4.3; 95% CI, 2.8-5.8), convenience of office location (1.5; 95% CI, 0.9-2.1), and online reviews (1.0; 95% CI, 0.7-1.3). There were no significant sex differences in importance scores. As participants’ age increased, they were more likely to value hospital reputation (P = .027), less likely to value a convenient location (P = .0005), less likely to care about wait time (P = .010), and less likely to care about insurance coverage (P = .0093).
Figure 2.
Importance scores for attributes.
Average scores across each category (physician reputation, physician qualification, hospital-related factors, and nonclinical factors) demonstrated that there was a significant difference between categories overall (P < .0001), with physician qualifications ranking most highly (average score, 10.4; 95% CI, 9.7-11.2), followed by hospital-related factors (average score, 8.6; 95% CI, 7.9-9.3). Physician reputation (average score, 6.4; 95% CI, 5.6-7.1) and nonclinical factors (average score, 5.1; 95% CI, 4.3-5.8) were significantly lower than the other categories but not statistically different from one another (P = .074).
Table 3 illustrates responses to the other non-BWS preference questions. Most (51.2%) patients were willing to wait 3 to 4 weeks for a laryngology appointment, whereas the majority (68.6%) had a 2-week or less wait time. Those participants who indicated less willingness to wait for an appointment on the stratified analysis had correspondingly higher importance scores placed on wait time as measured in the BWS exercise (P = .026), indicating reliability of the BWS exercises for assessing responses relative to wait time. The plurality of patients (37.2%) indicated that they would prefer a doctor with at least 6 to 10 years of experience; responses to this question correlated with BWS values pertaining to number of years the doctor has been in practice (P = .031), indicating reliability of the BWS exercises for assessing responses relative to doctor experience.
Table 3.
Other Preference Questions (N = 86).
| Characteristic | No. (%) |
|---|---|
| How long did you wait for this appointment? | |
| Less than 1 week | 29 (33.7) |
| 1-2 weeks | 30 (34.9) |
| 3-4 weeks | 22 (25.6) |
| More than 4 weeks | 5 (5.8) |
| What is the longest you would have been willing to wait? | |
| Less than 1 week | 6 (7.0) |
| 1-2 weeks | 21 (24.4) |
| 3-4 weeks | 44 (51.2) |
| More than 4 weeks | 15 (17.4) |
| How long did it take you to get to the office? | |
| Less than 15 minutes | 7 (8.1) |
| 15-30 minutes | 18 (20.9) |
| 30-60 minutes | 25 (29.1) |
| 1-2 hours | 26 (30.2) |
| More than 2 hours | 10 (11.6) |
| What is the longest time you would be willing to travel for this appointment? | |
| Less than 15 minutes | 1 (1.1) |
| 15-30 minutes | 9 (10.4) |
| 30-60 minutes | 30 (34.9) |
| 1-2 hours | 29 (33.7) |
| More than 2 hours | 17 (19.8) |
| I would want my doctor to have at least ___ years of experience | |
| 1-2 | 2 (2.3) |
| 3-5 | 31 (3.6) |
| 6-10 | 32 (37.2) |
| 10+ | 8 (9.3) |
| Doesn’t matter | 13 (15.1) |
| If I need surgery, I would want my surgeon to have performed my operation at least ___ times | |
| 10 | 12 (14.0) |
| 25 | 19 (22.1) |
| 50 | 17 (19.8) |
| 100 | 8 (9.3) |
| Greater than 100 | 17 (19.8) |
| Doesn’t matter | 13 (15.1) |
Discussion
Among a convenience sample of adult patients at an academic laryngology clinic, there are clear preferences for certain features related to choosing a laryngologist. Participants viewed the specialty qualifications of the physician as most important. In addition, there was a strong preference for hospital factors, including availability of advanced equipment, specialty focus of the hospital, and reputation of the hospital. Several of the nonclinical factors included on patient satisfaction surveys and online review sites—specifically, convenience of the office location and wait time for an appointment—were less important, on aggregate, relative to the other attributes. There was a correlation between these nonclinical factors’ importance scores and the more traditional multiple-choice questions, indicating reliability of BWS preferences. Certainly, for any individual patient, there may be a stronger or weaker preference for these factors; future larger studies could attempt to further segment this population into preference phenotypes (eg, “convenience seeking,”“reputation focused”) to more adequately target patients with health care services.
While it should not be surprising to otolaryngologists that patients have preferences related to care, the relative importance of various factors can have a significant impact on how patients choose physicians and how physicians deliver care. Indeed, research in orthopedics, plastic surgery, surgical oncology, dermatology, and primary care has used traditional surveys, discrete choice experiments, and more recently BWS to study how patients choose their surgeons, physicians, and hospital systems.10,13,14,18,26-28 Our results generally align with a BWS study by Ejaz et al18 evaluating how patients select cancer surgeons, which revealed that physician qualifications and hospital factors are more important than nonclinical factors. Interestingly, the most important factor in that study was surgeon caseload. This was not the case in our results, which demonstrated that caseload was the seventh most important of 13 factors, perhaps reflecting the often medical (rather than surgical) aspects of laryngology. Our results align with a systematic review of the main factors and sources of information patients use to choose their surgeon, in which the authors found that surgeon reputation and competency were the most valued individual attributes.28
Although the physician’s specialty qualifications were of the most importance to our study population, there was variability between the attributes within the rest of the physician qualification category (years of experience and number of surgeries performed), as has been found in previous research. While specialized training in laryngology was the most important attribute overall, the number of years of practice and the number of surgeries performed were much less important for patients. This is similar to what Waltzman et al26 found in their conjoint analysis of patient preferences for choosing a plastic surgeon, in that the number of years a surgeon has been in practice was only the fifth most important attribute, behind board certification, method of referral, distance from patient’s home, and office decor.5 The authors hypothesized that board certification may place enough confidence in the patient’s view of the surgeon’s training that the qualification outweighs the number of years of training. This is contrary to Abghari et al,27 who found that most patients prefer surgeons with 11 to 20 years of experience. In our study population, a plurality (45.5%) of patients preferred a doctor with greater than 5 years of experience, aligning more with patient preferences when choosing an orthopedic surgeon rather than a plastic surgeon. This may have implications for young faculty just establishing a practice, but it should be noted that other factors were even more important in this pilot cohort. Finally, it has been demonstrated that there is a paucity of accurate provider-specific information available through online review sites.29,30 In our data, this appears to be quite unimportant to respondents.
Prior literature has found that convenience of care is an important factor for patients when choosing a primary care physician.13 Our results, however, revealed relatively low importance scores associated with office location and wait time. One possible reason for this discrepancy could be the relative frequency of office visits to a primary care physician vs a subspecialist, which could influence how important nonclinical factors are for patients. In another study on consumer selection of a primary care physician, McGlone et al14 showed that convenience matters more for those with lower levels of education; they suggest that transportation issues may drive this. A high percentage of our group was college educated and may not have had these logistical concerns.
While there are few preference studies in otolaryngology relating to either the structural or outcome-related aspects of care, this information is important.31 Doctors and surgeons are prone to “preference misdiagnosis,” which can lead to suboptimal decision making.1,32 While this study is limited in scope to what drives patients to see a laryngologist, it provides insight into what patients prioritize during their evaluation. This can directly help physicians tailor access and practice patterns for patient satisfaction. However, the broader field of preference assessment can go beyond these structural aspects of care; this research is a stepping stone to further investigations of what patients prefer for treatment of their specific medical condition. We can imagine a scenario in which preference assessment undertaken during the workup of ear, nose, and throat conditions (eg, thyroid nodules or laryngeal cancer) helps guide treatment choices made together by physicians and patients. Fortunately, quantitative, econometric preference assessment with BWS is relatively easy and efficient and avoids many of the problems and biases associated with simple Likert scales and rating exercises.33,34
The study has limitations. We explicitly evaluated a convenience sample of patients presenting to a single outpatient laryngology clinic. This is a nonrandom sample and cannot be considered a reflection of the general population. Nevertheless, pilot studies such as this can provide in-depth preference data on the selected population, regardless of that population’s composition. Those seeking care at a tertiary referral center may attract a proportion of patients from around the region who are not concerned with wait time to get an appointment or convenience of the office location. They may also prioritize aspects of physicians that are less important to other patient subgroups. In addition, some patients may have been referred to a laryngologist without actually seeking this (eg, a patient following instructions from their primary care physician); nonetheless, patients with decision-making capacity have preferences that can be measured, regardless of how they have arrived in our office. As can be seen from our sample’s demographic characteristics, most patients were white and college educated, making it difficult to determine if our key outcomes differed based on demographic characteristics. Second, the sample size is not large enough to provide power to assess subgroups’ preferences. Future iterations beyond this pilot study should incorporate larger, more diverse, and perhaps randomized samples of patients seeking laryngology care.
Conclusion
This is the first study to our knowledge using best-worst scaling to identify the relative importance of factors patients consider when choosing a laryngologist. Physician specialty training and recommendations from referring physicians were the most important factors, while nonclinical factors such as wait time to get an appointment, convenience of office location, and online reviews were the least important. This information deepens health care providers’ knowledge of the patient decision-making process. Stated preference methodologies, including BWS, can help practitioners elucidate underlying preferences and make care more patient centered.
Author Contributions
Victoria Fischman, acquisition of data, analysis of data, interpretation of data, drafting, final approval; Eve Wittenberg, concept, design, interpretation of data, revision, final approval; Sungjin A. Song, concept, interpretation of data, drafting, revision, final approval; Molly N. Huston, interpretation of data, drafting, revision, final approval; Ramon A. Franco, concept, interpretation of data, revision, final approval; Phillip C. Song, concept, interpretation of data, revision, final approval; Matthew R. Naunheim, concept, design, acquisition of data, analysis of data, interpretation of data, drafting, revision, final approval.
Disclosures
Competing interests: None.
Sponsorships: None.
Funding source: None.
References
- 1. Naunheim MR, Wittenberg E, Shrime MG. Patient preference research in otolaryngology: what do patients want? JAMA Otolaryngol Head Neck Surg. 2017;143(10):971-972. [DOI] [PubMed] [Google Scholar]
- 2. Brennan PF, Strombom I. Improving health care by understanding patient preferences. J Am Med Inform Assoc. 1998;5(3):257-262. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Ruland CM. Decision support for patient preference-based care planning effects on nursing care and patient outcomes. J Am Med Inform Assoc. 1999;6(4):304-312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Shirk JD, Crespi CM, Saucedo JD, et al. Does patient preference measurement in decision aids improve decisional conflict? A randomized trial in men with prostate cancer. Patient. 2017;10(6):785-798. [DOI] [PubMed] [Google Scholar]
- 5. Bridges JFP. Stated preference methods in health care evaluation: an emerging methodological paradigm in health economics. Appl Health Econ Health Policy. 2003;2(4):213-224. [PubMed] [Google Scholar]
- 6. Flynn TN, Louviere JJ, Peters TJ, Coast J. Best-worst scaling: what it can do for health care research and how to do it. J Health Econ. 2007;26(1):171-189. [DOI] [PubMed] [Google Scholar]
- 7. Cohen SM. Self-reported impact of dysphonia in a primary care population: an epidemiological study. Laryngoscope. 2010;120(10):2022-2032. [DOI] [PubMed] [Google Scholar]
- 8. Tang B, Cai HD, Xie HL, Chen DY, Jiang SM, Jia L. Epidemiology of globus symptoms and associated psychological factors in China. J Dig Dis. 2016;17(5):319-324. [DOI] [PubMed] [Google Scholar]
- 9. Naunheim MR, Rathi VK, Naunheim ML, et al. What do patients want from otolaryngologists? A discrete choice experiment. Otolaryngol Neck Surg. 2017;157(4):618-624. [DOI] [PubMed] [Google Scholar]
- 10. Coast J, Salisbury C, de Berker D, et al. Preferences for aspects of a dermatology consultation. Br J Dermatol. 2006;155(2):387-392. [DOI] [PubMed] [Google Scholar]
- 11. Dijs-Elsinga J, Otten W, Versluijs MM, et al. Choosing a hospital for surgery: the importance of information on quality of care. Med Decis Mak. 2010;30(5):544-555. [DOI] [PubMed] [Google Scholar]
- 12. Marang-van de Mheen PJ, Dijs-Elsinga J, Otten W, et al. The relative importance of quality of care information when choosing a hospital for surgical treatment: a hospital choice experiment. Med Decis Mak. 2011;31(6):816-827. [DOI] [PubMed] [Google Scholar]
- 13. Bornstein BH, Marcus D, Cassidy W. Choosing a doctor: an exploratory study of factors influencing patients’ choice of a primary care doctor. J Eval Clin Pract. 2000;6(3):255-262. [DOI] [PubMed] [Google Scholar]
- 14. McGlone TA, Butler ES, McGlone VL. Factors influencing consumers’ selection of a primary care physician. Health Mark Q. 2002;19(3):21-37. [DOI] [PubMed] [Google Scholar]
- 15. Bozic KJ, Kaufman D, Chan VC, Caminiti S, Lewis C. Factors that influence provider selection for elective total joint arthroplasty. Clin Orthop. 2013;471(6):1865-1872. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Wilson CT, Woloshin S, Schwartz LM. Choosing where to have major surgery: who makes the decision? Arch Surg. 2007;142(3):242-246. [DOI] [PubMed] [Google Scholar]
- 17. Cheung KL, Wijnen BFM, Hollin IL, et al. Using best-worst scaling to investigate preferences in health care. Pharmacoeconomics. 2016;34(12):1195-1209. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Ejaz A, Spolverato G, Bridges JF, Amini N, Kim Y, Pawlik TM. Choosing a cancer surgeon: analyzing factors in patient decision making using a best-worst scaling methodology. Ann Surg Oncol. 2014;21(12):3732-3738. [DOI] [PubMed] [Google Scholar]
- 19. Coast J, Horrocks S. Developing attributes and levels for discrete choice experiments using qualitative methods. J Health Serv Res Policy. 2007;12(1):25-30. [DOI] [PubMed] [Google Scholar]
- 20. Coast J, Al-Janabi H, Sutton EJ, et al. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ. 2012;21(6):730-741. [DOI] [PubMed] [Google Scholar]
- 21. Bridges JFP, Hauber AB, Marshall D, et al. Conjoint analysis applications in health—a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value Health. 2011;14(4):403-413. [DOI] [PubMed] [Google Scholar]
- 22. Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making: a user’s guide. Pharmacoeconomics. 2008;26(8):661-677. [DOI] [PubMed] [Google Scholar]
- 23. Lancsar E, Fiebig DG, Hole AR. Discrete choice experiments: a guide to model specification, estimation and software. Pharmacoeconomics. 2017;35(7):697-716. [DOI] [PubMed] [Google Scholar]
- 24. Whitty JA, Oliveira Gonçalves AS. A systematic review comparing the acceptability, validity and concordance of discrete choice experiments and best-worst scaling for eliciting preferences in healthcare. Patient. 2018;11(3):301-317. [DOI] [PubMed] [Google Scholar]
- 25. Sawtooth Software. The CBC System for Choice-Based Conjoint Analysis Version 8. Sawtooth Software; 2013. [Google Scholar]
- 26. Waltzman JT, Scholz T, Evans GRD. What patients look for when choosing a plastic surgeon: an assessment of patient preference by conjoint analysis. Ann Plast Surg. 2011;66(6):643-647. [DOI] [PubMed] [Google Scholar]
- 27. Abghari MS, Takemoto R, Sadiq A, Karia R, Phillips D, Egol KA. Patient perceptions and preferences when choosing an orthopaedic surgeon. Iowa Orthop J. 2014;34:204-208. [PMC free article] [PubMed] [Google Scholar]
- 28. Yahanda AT, Lafaro KJ, Spolverato G, Pawlik TM. A systematic review of the factors that patients use to choose their surgeon. World J Surg. 2016;40(1):45-55. [DOI] [PubMed] [Google Scholar]
- 29. Ryan T, Specht J, Smith S, DelGaudio JM. Does the Press Ganey Survey correlate to online health grades for a major academic otolaryngology department? Otolaryngol Head Neck Surg. 2016;155(3):411-415. [DOI] [PubMed] [Google Scholar]
- 30. Calixto NE, Chiao W, Durr ML, Jiang N. Factors impacting online ratings for otolaryngologists. Ann Otol Rhinol Laryngol. 2018;127(8):521-526. [DOI] [PubMed] [Google Scholar]
- 31. Naunheim MR, Randolph GW, Shin JJ. Evidence-based medicine in otolaryngology part XII: assessing patient preferences [published online September 8, 2020]. Otolaryngol Head Neck Surg. [DOI] [PubMed] [Google Scholar]
- 32. Mulley AG, Trimble C, Elwyn G. Stop the silent misdiagnosis: patients’ preferences matter. BMJ. 2012;345:e6572. [DOI] [PubMed] [Google Scholar]
- 33. Flynn TN. Valuing citizen and patient preferences in health: recent developments in three types of best-worst scaling. Expert Rev Pharmacoecon Outcomes Res. 2010;10(3):259-267. [DOI] [PubMed] [Google Scholar]
- 34. Louviere JJ, Flynn TN, Marley AAJ. Best-Worst Scaling: Theory, Methods and Applications. Cambridge University Press; 2015. [Google Scholar]