Abstract
Background:
Patient utilization of public reporting has been suboptimal despite attempts to encourage use. Lack of utilization may be due to discordance between reported metrics and what patients want to know when making healthcare choices.
Objectives:
To identify measures of quality that individuals want presented in public reporting and explore factors associated with researching healthcare.
Research Design:
Patient interviews and focus groups were conducted to develop a survey exploring the relative importance of various healthcare measures.
Subjects:
Interviews and focus groups conducted at local outpatient clinics. Survey administered nationally on anonymous digital platform.
Measures:
Likert scale responses were compared using tests of central tendency. Rank-order responses were compared using analysis of variance testing. Associations with binary outcomes were analyzed using multivariable logistic regression.
Results:
Overall, 4,672 responses were received (42.0% response rate). Census balancing yielded 2,004 surveys for analysis. Measures identified as most important were hospital reputation (considered important by 61.9%), physician experience (51.5%), and primary care recommendations (43.2%). Unimportant factors included guideline adherence (17.6%) and hospital academic affiliation (13.3%, p<0.001 for all compared to most important factors). Morbidity and mortality outcome measures were not among the most important factors. Patients were unlikely to rank outcome measures as the most important factors in choosing healthcare providers, irrespective of age, gender, educational status, or income.
Conclusions:
Patients valued hospital reputation, physician experience, and primary care recommendations while publicly reported metrics like patient outcomes were less important. Public quality reports contain information that patients perceive to be of relatively low value, which may contribute to low utilization.
Introduction:
Over the past two decades, there has been a push towards increased information transparency and public reporting within healthcare.1 Goals of enhanced transparency in healthcare quality reporting include greater provider accountability, increased quality measures, improved outcomes, and ability for patients to make informed decisions regarding where to seek care.2-4 Despite the importance of healthcare quality reporting, the emphasis on transparency has led to a substantial increase in publically available data with varying quality metrics of unclear significance.5 Furthermore, patient utilization of healthcare quality reporting has been suboptimal despite attempts to encourage its use.6
Several theories regarding the underutilization of publicly available healthcare quality reports by patients include an excess of information made available to patients, the lack of a comprehensive source of information, or available information does not represent patient preferences.7 While efforts have been made to address the first two issues5, concerns persist regarding patient preferred metrics to inform healthcare decisions. Therefore, potential discordance between reported quality metrics and the information individuals value when making healthcare choices represents an area for improvement. Previous work has shown that patients value access to information regarding quality of care as well as comparisons between sources of care.7-9 However, specific metrics and how patients prioritize those metrics when choosing providers or hospitals remains unclear.
As the push for transparency in healthcare quality reporting continues, improvements in patient utilization of publicly available reports remain essential. However, causes of patient underutilization of available data remain poorly studied. Therefore, evaluating specific patient preferences may provide tools to improve current healthcare quality reporting and utilization. The objectives of this study were to (1) quantify how often individuals research healthcare, (2) identify hospital- and physician-level measures of quality that individuals want presented in public reporting, and (3) explore factors that may drive individuals to research healthcare.
Methods:
Identification of Patient-Centered Concepts of Healthcare Quality
Semi-structured interviews were conducted with patients in general surgery clinics to identify preliminary concepts, categories, and language for subsequent survey development. Next, focus groups were used to complete exploration of hospital quality concepts. Patients were excluded if legally blind, pregnant, or seeking consultation for potential surgery. These exclusion criteria were due to logistical constraints regarding informational material and concern for vulnerable populations. Physicians in target clinics granted approval to contact their patients for the study. All participants were provided written informed consent and compensated for their time with a $40 Visa gift card prior to participation.
During semi-structured interviews, participants were asked to describe the process they used to select a hospital for surgery, to identify hospital factors that influenced their choice, and to discuss the utility of both available and hypothetical hospital quality measures. During focus groups, participants were asked to identify and suggest hospital factors and quality measures that influenced a patient’s hospital choice for surgery. Interviews were conducted until thematic saturation was achieved.
Survey Development
Themes identified were subsequently built into a survey designed to explore patient-centered concepts of healthcare and surgical quality. Concepts repeatedly identified or endorsed during patient interviews were written as survey questions. Interview results indicated that insurance coverage was a dominant factor in choosing where to receive healthcare (i.e., patients would not consider going to a facility that was not covered by their insurance, regardless of quality). As such, questions were constructed to identify those factors that patients would consider in choosing between hospitals and providers that accepted their insurance. For the purposes of the survey, questions were grouped into hospital factors, physician factors, and outcome measures. Respondents were asked to first select those factors they would consider in choosing where to receive care. Subsequently, respondents were asked to assess the importance of all factors they would consider in choosing where to receive care on a 5 point Likert Scale (Very Unimportant to Very Important). Finally, respondents were asked to assess the relative importance of the most important factors. Respondents were also asked general questions about their health history (e.g., previous hospitalizations, previous attempts at researching healthcare) and functional status (4-item PROMIS Short Form for Physical Function).10 After initial construction of the survey, pretest cognitive interviews were conducted with additional patients in general surgery clinics to assess survey coherence, balance, and clarity. The survey was iteratively revised and re-tested in additional patient volunteers prior to online pilot testing.
Outcomes
Two outcomes of interest were assessed: previous attempts at researching healthcare and prioritization of outcome measures in choosing where to receive healthcare. Respondents were considered to prioritize outcome measures if outcome measures were among the three most important factors identified by that patient in choosing where to receive healthcare.
Qualitative Analysis
A researcher trained in qualitative research analyzed the interview transcripts. QDA Miner 4 (Provalis Research, Montreal, Quebec, Canada) was used to facilitate coding. Semi-structured interview and focus group participant responses were analyzed using thematic analysis, a systematic search for themes, patterns, and repetitions throughout and across the interviews.11 Using a line-by-line approach,12 a preliminary codebook was developed until saturation was reached. Two independent analysts then applied the codes to all transcripts (including the transcripts used for codebook development).
Statistical Analysis
The final survey was disseminated via the SurveyGizmo platform (Boulder, CO) to preexisting, standing survey panels. Responses were collected until 2,000 census-balanced responses were available for analysis. Due to the structure of the public survey platform, the process of census optimization involved continued distribution of the survey until a minimum number of responses was achieved from each relevant demographic subgroup. A random sample was taken from any subgroup that was over-sampled during the survey dissemination to create the final survey dataset. Response rate was calculated based on the total survey responses, not only those available for analysis. Likert scale responses were compared using non-parametric tests of central tendency. Rank order responses were compared using analysis of variance testing adjusted for multiple comparisons. Multivariable logistic regression models were estimated to examine the association between patient health history, demographics, functional status, and both previous attempts to research healthcare and prioritization of outcome measures in choosing where to receive care. Point estimates are reported without confidence intervals, and level of significant was set to 0.05. Data analyses were performed using STATA 15.1 (StataCorp LP, College Station, TX). This study was approved by the Northwestern University Institutional Review Board.
Results:
Patient Interviews and Focus Groups
In order to reach saturation, ten semi-structured interviews and three focus groups were conducted. Focus groups included two, two, and seven participants, respectively. This yielded several factors that patients consider both important and unreliable in assessing where to receive care. Factors that patients most frequently identified as important drivers included hospital reputation or ranking; hospital appearance and cleanliness; hospital location; referrals by primary care physicians, friends, or family; physician and staff personality; physician credentials; and the quality of follow up after care. Qualities that were often mentioned but generally considered unreliable or unnecessary by patients included infection rates, hand washing, and complications rates. Representative hospital factors, physician factors, and outcome measures derived from these encounters were then used to construct a survey to quantitatively explore these themes.
Survey Cohort
Online surveys were distributed to 11,125 individuals with 4,672 completed responses (42.0% response rate). Census optimization yielded 2,004 surveys for analysis. Overall, 66.0% of respondents were under the age of 45 and 51.1% were female. More than two-thirds (68.5%) had undergone surgery and 69.7% had been admitted to a hospital. Most respondents reported previously researching healthcare in some way (60.5%), with 91.5% of those that had previously researched healthcare stating the research helped them to make their healthcare decision. Additional cohort characteristics can be found in Table 1.
Table 1.
Respondent Characteristics (N=2,004)
| Variable | n, (%) |
|---|---|
| Age | |
| <34 | 778 (38.8) |
| 35-44 | 544 (27.2) |
| 45-54 | 397 (19.8) |
| >55 | 285 (14.2) |
| Gender | |
| Female | 1023 (51.1) |
| Male | 966 (48.2) |
| Other/Prefer not to say | 15 (0.7) |
| Region | |
| Northeast | 457 (22.8) |
| Southeast | 462 (23.1) |
| Midwest | 475 (23.7) |
| Southwest | 202 (10.1) |
| West | 408 (20.4) |
| Ethnicity | |
| Non-Hispanic White | 1464 (73.1) |
| Non-Hispanic Black | 219 (10.9) |
| Hispanic/Latino | 141 (7.0) |
| Asian | 113 (5.6) |
| Other/Prefer Not to Say | 67 (3.4) |
| Education | |
| Advanced (MD, PhD, MS) | 355 (17.7) |
| Bachelor’s Degree | 530 (26.5) |
| Associates Degree | 208 (10.4) |
| High School Diploma | 911 (45.5) |
| Income | |
| >$100,000 | 531 (26.5) |
| $50,000-$99,999 | 622 (31.0) |
| <$50,000 | 851 (42.5) |
| Health History | |
| Has Undergone Surgery | 1373 (68.5) |
| Has Been Admitted >1 Night | 1397 (69.7) |
| Health Status* | |
| A (4-13) | 455 (22.2) |
| B (14-19) | 524 (26.1) |
| C (20) | 1035 (51.7) |
| Previous Healthcare Research | |
| Has researched healthcare | 1213 (60.5) |
| 1071 (53.4) | |
| Hospital Websites | 725 (36.2) |
| Online Rankings | 422 (21.1) |
| Yelp | 163 (8.1) |
| Insurance Sites | 34 (1.7) |
| Other | 15 (0.8) |
| Researched helped decision** | 1150 (91.5) |
| Why No Research | |
| No Need for Care | 227 (11.3) |
| Have Relationship | 139 (6.9) |
| Relied on Family | 39 (2.0) |
| Relied on PCP | 129 (6.4) |
| Emergency | 98 (4.9) |
| Insurance Only | 92 (4.6) |
| Other | 67 (3.3) |
Based on four item PROMIS measure (tertiles with a perfect score 20/20 being group C)
Denominator of 1213 (number who performed research)
Choosing Where to Receive Care
Among hospital factors, respondents most often considered the reputation of the hospital (61.9%), the location of the hospital (50.6%), and primary physician recommendations (53.2%) to be important factors in choosing between hospitals that accept their insurance. The least important hospital factors included university affiliations (85.0% would not consider), familiarity with faculty (82.6%), hospital amenities (76.9%), and the ability for family to stay near the hospital (76.4%). Relative importance of additional hospital factors can be found in Table 2.
Table 2:
Hospital Factors
| Very Important n, (%) |
Important n, (%) |
Neutral n, (%) |
Unimportant n, (%) |
Very Unimportant n, (%) |
Would Not Consider n, (%) |
|
|---|---|---|---|---|---|---|
| Location of the hospital | 509 (25.4) | 504 (25.2) | 64 (3.2) | 1 (0.1) | 5 (0.3) | 921 (46.0) |
| Reputation of the hospital | 919 (45.9) | 321 (16.0) | 15 (0.8) | 2 (0.1) | 3 (0.2) | 744 (37.1) |
| How often your specific problem is taken care of at that hospital | 481 (24) | 253 (12.6) | 26 (1.3) | 1 (0.1) | 3 (0.2) | 1240 (61.9) |
| Recommendations from family and friends regarding the hospital | 324 (16.2) | 431 (21.5) | 71 (3.5) | 3 (0.2) | 2 (0.1) | 1173 (58.5) |
| Recommendations from your primary doctor regarding the hospital | 684 (34.1) | 382 (19.1) | 43 (2.2) | 0 (0.0) | 2 (0.1) | 893 (44.6) |
| Ability of family to stay near the hospital | 195 (9.7) | 220 (11.0) | 55 (2.7) | 3 (0.2) | 0 (0.0) | 1531 (76.4) |
| Hospital amenities (television, WiFi internet access, etc) | 163 (8.1) | 228 (11.4) | 69 (3.4) | 2 (0.1) | 1 (0.1) | 1541 (76.9) |
| Private hospital rooms | 322 (16.1) | 273 (13.6) | 44 (2.2) | 6 (0.3) | 0 (0.0) | 1359 (67.8) |
| Hospital ranking on ratings website | 347 (17.3) | 300 (15.0) | 35 (1.8) | 4 (0.2) | 1 (0.1) | 1317 (65.7) |
| Hospital is affiliated with a major university | 115 (5.7) | 153 (7.6) | 26 (1.3) | 4 (0.2) | 2 (0.1) | 1704 (85.0) |
| Familiarity with faculty and staff working at the hospital | 124 (6.2) | 167 (8.3) | 56 (2.8) | 2 (0.1) | 0 (0.0) | 1655 (82.6) |
No factors were considered important by 122 (6.1%) patients
Among physician factors, the most important were years of experience (51.1% considered important), primary physician recommendations (49.2%), and overall satisfaction of previous patients (48.1%). The least important physician factors included physician race (93.0% would not consider), the presence of trainees (87.1%), physician gender (86.2%), and how well the physician follows national guidelines (80.6%). Additional information on the importance of physician factors can be found in Table 3.
Table 3:
Physician Factors
| Very Important n, (%) |
Important n, (%) |
Neutral n, (%) |
Unimportant n, (%) |
Very Unimportant n, (%) |
Would Not Consider n, (%) |
|
|---|---|---|---|---|---|---|
| Years of experience | 617 (30.8) | 406 (20.3) | 30 (1.5) | 1 (0.1) | 1 (0.1) | 949 (47.4) |
| Reputation of the doctor’s medical school or residency training program | 320 (16.0) | 244 (12.2) | 24 (1.2) | 2 (0.1) | 1 (0.1) | 1413 (70.5) |
| Overall satisfaction of previous patients | 621 (31.0) | 342 (17.1) | 30 (1.5) | 3 (0.2) | 1 (0.1) | 1007 (50.3) |
| How well the doctor follows national guidelines | 244 (12.2) | 125 (6.2) | 18 (0.9) | 1 (0.1) | 1 (0.1) | 1615 (80.6) |
| Recommendations from family and friends regarding the doctor | 382 (19.1) | 442 (22.1) | 58 (2.8) | 2 (0.1) | 2 (0.1) | 1118 (55.8) |
| Recommendations from your primary doctor regarding the doctor | 613 (30.6) | 373 (18.6) | 27 (1.4) | 2 (0.1) | 2 (0.1) | 987 (49.3) |
| Doctor’s race | 63 (3.1) | 49 (2.5) | 18 (0.9) | 8 (0.4) | 2 (0.1) | 1864 (93.0) |
| Doctor’s gender | 110 (5.5) | 127 (6.3) | 36 (1.8) | 1 (0.1) | 3 (0.2) | 1727 (86.2) |
| Doctor’s scheduling flexibility | 451 (22.5) | 412 (20.6) | 31 (1.6) | 5 (0.3) | 1 (0.1) | 1104 (55.1) |
| A doctor in training will not participate in my care | 135 (6.7) | 81 (4.0) | 29 (1.5) | 10 (0.5) | 3 (0.2) | 1746 (87.1) |
| Personal familiarity with the doctor | 230 (11.5) | 279 (13.9) | 86 (4.3) | 4 (0.2) | 2 (0.1) | 1403 (70.0) |
| Doctor’s rating on ratings website | 331 (16.5) | 346 (17.3) | 36 (1.8) | 3 (0.2) | 2 (0.1) | 1286 (64.2) |
No factors were considered important by 109 (5.4%) patients
Among outcome measures, only how quickly patients feel back to normal after a hospital stay (43.3%) and how quickly patients are fully functional after a hospital stay (43.0%) were considered important by more than 40% of patients. Less important outcome measures included the risk of requiring temporary nursing home care (81.3% would not consider), the risk that their problem is not fixed by the hospitalization (64.8%), the risk of death (64.3%), and the risk of readmission (64.2%). Additional information on the importance of outcome measures can be found in Table 4.
Table 4:
Outcome Factors
| Very Important n, (%) |
Important n, (%) |
Neutral n, (%) |
Unimportant n, (%) |
Very Unimportant n, (%) |
Would Not Consider n, (%) |
|
|---|---|---|---|---|---|---|
| Risk of death following treatment | 602 (30.0) | 102 (5.1) | 8 (0.4) | 1 (0.1) | 2 (0.1) | 1289 (64.3) |
| Risk of requiring temporary nursing home care after treatment | 204 (10.2) | 146 (7.3) | 21 (1.1) | 3 (0.2) | 1 (0.1) | 1629 (81.3) |
| Risk of having a complication (example: infection, blood clot) | 685 (34.2) | 181 (9.0) | 19 (1.0) | 1 (0.1) | 1 (0.1) | 1117 (55.7) |
| How quickly patients feel back to normal after a hospital stay | 476 (23.8) | 391 (19.5) | 30 (1.5) | 0 (0.0) | 0 (0.0) | 1107 (55.2) |
| How often patients receive all appropriate medications after a hospital stay | 467 (23.3) | 260 (13.0) | 24 (1.2) | 0 (0.0) | 0 (0.0) | 1253 (62.5) |
| How often patients receive appropriate follow up appointments after a hospital stay | 401 (20.0) | 331 (16.5) | 29 (1.5) | 3 (0.2) | 1 (0.1) | 1239 (61.8) |
| How quickly patients are fully functional after a hospital stay | 536 (26.8) | 325 (16.2) | 37 (1.9) | 2 (0.1) | 1 (0.1) | 1103 (55.0) |
| Risk of having to return to the hospital for the same problem within the next month | 467 (23.3) | 222 (11.2) | 28 (1.4) | 1 (0.2) | 0 (0.0) | 1286 (64.2) |
| Risk that my problem is not completely fixed by the hospitalization | 479 (23.9) | 207 (10.3) | 17 (0.9) | 3 (0.2) | 0 (0.0) | 1298 (64.8) |
No factors were considered important by 236 (11.8%) patients
Respondents were then asked to rank the relative importance of eight factors in choosing where to receive care: hospital location, hospital reputation, recommendations from primary physicians, physician experience, overall satisfaction of previous patients, risk of death, risk of complications, and how quickly patients were fully functional after hospitalization. These eight factors were chosen based on being the most frequently selected factors in pilot testing. There were no significant differences in the ranks of any factors, with the mean and median rank of all eight factors being between 4.0 and 4.9 (Supplemental Digital Table 1).
Factors Associated with Previous Healthcare Research and Prioritizing Outcome Measures
Overall, 60.5% of respondents reported previously researching healthcare online. Respondents were more likely to have previously performed research if <35 years old (aOR 2.23, 95% CI: 1.65-3.00, P<0.001; vs >55 years old), female (aOR 1.30 vs male, 95% CI: 1.07-1.58, P=0.009), Hispanic or Latino (aOR 1.94, 95%CI: 1.30-2.89; vs non-Hispanic White), had advanced degrees (aOR 2.09, 95%CI: 1.53-2.86, P<0.001; vs those with high school or less education), or had income ≥$100,000 (aOR 1.75, 95%CI: 1.33-2.30, P<0.001; vs income <$50,000). Respondents were also more likely to endorse previously researching healthcare if they had previously had surgery (aOR 1.56, 95%CI: 1.25-1.96, P<0.001), previously been admitted to the hospital (aOR 1.70, 95%CI: 1.36-2.14, P<0.001), or had relatively low function status (aOR 1.64, 95%CI: 1.27-2.12, P<0.001; vs high functional status; Table 5).
Table 5:
Multivariable Analysis of Factors Associated with Previous Healthcare Research and Prioritization of Outcome Measures
| Performed Research | Prioritize Outcome Measures | |||||
|---|---|---|---|---|---|---|
| Unadj Rate |
OR (95% CI) | P value | Unadj Rate |
OR (95% CI) | P value | |
| Overall | 60.5 | 38.7 | ||||
| Demographics | ||||||
| Age Group | ||||||
| <35 | 61.4 | 2.23 (1.65-3.00) | <0.001 | 40.6 | 1.04 (0.77-1.40) | 0.805 |
| 35-44 | 66.5 | 2.24 (1.64-3.07) | <0.001 | 36.6 | 0.86 (0.63-1.18) | 0.350 |
| 45-54 | 59.2 | 1.84 (1.32-2.55) | <0.001 | 37.6 | 0.92 (0.66-1.28) | 0.618 |
| ≥55 | 48.4 | 1.0 Ref | 39.0 | 1.0 Ref | ||
| Gender | ||||||
| Male | 57.6 | 1.0 Ref | 38.2 | 1.0 Ref | ||
| Female | 63.8 | 1.30 (1.07-1.58) | 0.009 | 39.3 | 0.98 (0.81-1.19) | 0.841 |
| Other/Prefer Not | 26.7 | 0.59 (0.17-2.01) | 0.397 | 30.8 | 0.45 (0.12-1.66) | 0.230 |
| Ethnicity | ||||||
| Non-Hispanic White | 60.1 | 1.0 Ref | 37.9 | 1.0 Ref | ||
| Non-Hispanic Black | 59.8 | 1.27 (0.93-1.75) | 0.136 | 42.3 | 1.22 (0.89-1.66) | 0.210 |
| Hispanic/Latino | 72.3 | 1.94 (1.30-2.89) | 0.001 | 35.7 | 0.89 (0.60-1.32) | 0.548 |
| Asian | 56.6 | 0.86 (0.56-1.32) | 0.492 | 35.7 | 0.92 (0.59-1.43) | 0.703 |
| Unknown | 53.7 | 0.93 (0.51-1.66) | 0.795 | 54.1 | 2.18 (1.28-3.73) | 0.004 |
| Location | ||||||
| Northeast | 63.9 | 1.30 (0.98-1.72) | 0.074 | 38.8 | 1.02 (0.77-1.36) | 0.871 |
| Southeast | 63.0 | 1.36 (1.03-1.80) | 0.030 | 39.2 | 1.04 (0.78-1.38) | 0.793 |
| Midwest | 52.4 | 1.0 Ref | 38.6 | 1.0 Ref | ||
| Southwest | 64.4 | 1.47 (1.01-2.12) | 0.042 | 41.2 | 1.14 (0.80-1.62) | 0.478 |
| West | 61.5 | 1.34 (1.01-1.80) | 0.049 | 36.8 | 0.92 (0.69-1.24) | 0.601 |
| Education | ||||||
| Advanced Degree | 73.2 | 2.09 (1.53-2.86) | <0.001 | 36.7 | 0.90 (0.66-1.21) | 0.482 |
| Bachelors | 69.1 | 1.94 (1.50-2.50) | <0.001 | 40.8 | 1.07 (0.84-1.38) | 0.571 |
| Associates | 62.0 | 1.50 (1.08-2.08) | 0.015 | 34.7 | 0.82 (0.59-1.15) | 0.247 |
| High School or Less | 50.3 | 1.0 Ref | 39.1 | 1.0 Ref | ||
| Income | ||||||
| ≥100,000 | 60.6 | 1.75 (1.33-2.30) | <0.001 | 36.9 | 1.02 (0.78-1.34) | 0.877 |
| 50k-49.9k | 66.2 | 1.70 (1.34-2.16) | <0.001 | 41.9 | 1.24 (0.98-1.57) | 0.075 |
| <50k | 50.1 | 1.0 Ref | 37.4 | 1.0 Ref | ||
| Health History | ||||||
| Previously Had Surgery | ||||||
| No | 48.8 | 1.0 Ref | 40.0 | 1.0 Ref | ||
| Yes | 65.9 | 1.56 (1.25-1.96) | <0.001 | 38.1 | 0.96 (0.77-1.21) | 0.753 |
| Previous Admission | ||||||
| No | 47.9 | 1.0 Ref | 40.8 | 1.0 Ref | ||
| Yes | 66.0 | 1.70 (1.36-2.14) | <0.001 | 37.7 | 0.91 (0.72-1.13) | 0.387 |
| Health Scale | ||||||
| A (Unhealthy) | 64.3 | 1.64 (1.27-2.12) | <0.001 | 39.1 | 1.03 (0.81-1.32) | 0.801 |
| B | 66.8 | 1.64 (1.29-2.09) | <0.001 | 36.7 | 0.93 (0.74-1.18) | 0.545 |
| C (Healthy) | 55.8 | 1.0 Ref | 39.5 | 1.0 Ref | ||
Outcome measures were prioritized (among top three choices in factor rankings) by 38.7% of respondents. There were no significant demographic or health-related factors that predicted respondents prioritizing outcome measures in choosing where to receive healthcare (Table 5).
Discussion:
In this study, we used a national survey of internet users to define how often individuals research healthcare and what factors individuals valued in making decisions about where to receive care. While more than half of survey respondents had previously researched healthcare, a large number of these cited simple online searches without utilization of validated measures. While respondents were more likely to consider hospital reputation and recommendations from primary care physicians, no single factor was considered important by more than two thirds of respondents. While individuals were more likely to perform research if younger, more educated, or had previous experience with the healthcare system, there were no factors associated with prioritizing hospital quality or outcome measures such as complication or mortality rates.
The most striking result of this study is the relative indifference with which respondents viewed measures of healthcare quality that are commonly thought to be the important, such as complication and death rates. Patients were even less likely to consider risk of inaccurate medical reconciliations or readmissions, both of which are commonly used quality measures.13, 14 While previous studies have shown relatively low but slowly increasing utilization rates of quality-based healthcare research overall,15, 16 this study provides granular information on the patient preferences that underlie decision making. While some have hypothesized that the complexity and presentation of quality measures may make it challenging for patients to navigate this space and interpret publicly available data,17 these results imply that patients may not value the underlying raw numbers. This conclusion is further supported by the lack of demographic factors (e.g., education, income, previous healthcare) associated with prioritization of tangible healthcare outcome measures, which implies that prioritization of healthcare quality or outcome measures is not simply a matter of education or experience.
These results were supported on a smaller scale through our qualitative results. Throughout the study, participants used personal experiences to estimate hospital quality. For example, multiple participants believed third-party measures of infection rates and hand washing were unreliable. Instead, participants interested in postoperative complication rates looked directly to their surgeon, primary care provider, or family members with more experience in healthcare for answers.
An additional interesting finding of this study is what the general population considers researching healthcare. While more than 60% of respondents stated they had researched healthcare and more than 90% of those said the research helped their decision, a large amount of this research appeared to involve simple online searches or exploring hospital websites. These results imply that patients may not even conceptualize “researching where to receive care” in the same way as those designing hospital rankings and quality measures, further highlighting the chasm that must be bridged in order to increase thorough, patient-based interpretation of healthcare quality data.
While these results may be discouraging to those working to develop and disseminate healthcare quality measures and hospital rankings, they do provide some insight into steps that may improve utilization of healthcare quality measures. First, the factor most commonly selected by respondents was Hospital Reputation, which is difficult to quantify but likely at least partially derived from hospital quality measures and rankings (e.g., U.S. News Rankings). Additional research on how patients ascertain or conceptualize hospital reputation are warranted, and development of reporting systems focused on relevant, patient-centered information may improve patient utilization of quality data.18, 19 Second, this study highlights the integral role of primary care physicians in guiding the decisions of patients that may be uncomfortable interpreting data primarily. The role of patients as consumers in the traditional sense has been extensively discussed in the literature, and primary care physicians clearly play a role in helping their patients navigate the complicated healthcare market.20-22 Educational efforts aimed at primary care physicians to encourage their utilization of quality measures when recommending where to receive complex care may improve indirect dissemination of these data to patients. Finally, this study highlights that some measures commonly employed to describe hospital quality (e.g., readmission rates) may need to be reframed in more patient-centered or functional terms in order to improve patient use of quality data. This may involve incorporation of measures that the public finds most compelling (e.g., return to functionality after illness), simple rewording of how the existing metrics are presented, and additional educational efforts to explain the importance of certain measures. The mechanism of dissemination for such research should be further investigated, but could include government sponsored report cards of continued outreach from private research enterprises.
This study must be viewed in light of its limitations. First, this cross-sectional study can only explore associations and cannot identify causal relationships. Second, the online survey mechanism limits responses to only internet users, and thus may not reflect the entirety of the population. However, we believe this population would bias the study towards younger and potentially more technologically savvy respondents, and thus these results likely overestimate how much healthcare research is performed. Third, we did not specifically define the concept of “previous healthcare research” which may have artificially inflated the number of respondents reporting that outcome. However, we believe allowing respondents to classify what they considered to be “research” was equally interesting and demonstrates a fundamental gap in understanding among some respondents. Fourth, the relatively low rates of individuals marking factors as “unimportant” implies that many individuals may not have distinguished between “would not consider” and “unimportant,” which may limit the distinction between the two survey responses. However, this is unlikely to affect final models as the outcomes were those that considered certain factors important or very important. Finally, we were unable to provide any more than a rudimentary exploration of the role of insurance coverage in healthcare decisions. This is intuitive given the high out-of-pocket cost of healthcare in the United States, but future research should continue to explore patient-based healthcare research in the context of the American insurance structure.
Conclusions:
While more than half of individuals may have performed some amount of healthcare research, this research often does not include robust measures of healthcare quality. This may be at least partially driven by differences between what data patients prioritize in making healthcare decisions and what is presented by healthcare quality sites and rankings. Development of reporting systems focused on relevant, patient-centered information may improve patient utilization of quality data.
Supplementary Material
Acknowledgments
Funding: Project supported by the Agency for Healthcare Research and Quality grant (R21HS021857; PI: Bilimoria) entitled “Engaging Patients and Hospitals to Expand Public Reporting in Surgery.” RJE, TYK, and DBH were supported by a postdoctoral research fellowship (Agency for Healthcare Research and Quality [AHRQ] 5T32HS000078). RPM is supported by the Agency for Healthcare Quality (K12HS026385) and an Institutional Research Grant from the American Cancer Society (IRG-18-163-24).The American College of Surgeons as an organization had no role in the design and conduct of the study; analysis and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Views expressed in this work represent those of the authors only.
Footnotes
Disclosures: the authors report no conflicts of interest, financial or otherwise, related to this work.
Meeting Information: Results were presented at the 14th Annual Academic Surgical Congress, February 5-7, 2019 in Houston, TX.
References
- 1.Rothberg MB, Morsi E, Benjamin EM, et al. Choosing the best hospital: the limitations of public quality reporting. Health Aff (Millwood). 2008; 27(6): 1680–7. [DOI] [PubMed] [Google Scholar]
- 2.Rosenthal GE, Quinn L, and Harper DL. Declines in hospital mortality associated with a regional initiative to measure hospital performance. Am J Med Qual. 1997; 12(2): 103–12. [DOI] [PubMed] [Google Scholar]
- 3.Schauffler HH and Mordavsky JK. Consumer reports in health care: do they make a difference? Annu Rev Public Health. 2001; 22: 69–89. [DOI] [PubMed] [Google Scholar]
- 4.Schmaltz SP, Williams SC, Chassin MR, et al. Hospital performance trends on national quality measures and the association with Joint Commission accreditation. J Hosp Med. 2011; 6(8): 454–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Pronovost PJ, Wu AW, and Austin JM. Time for Transparent Standards in Quality Reporting by Health Care Organizations. JAMA. 2017; 318(8): 701–702. [DOI] [PubMed] [Google Scholar]
- 6.Yang A, Chimonas S, Bach PB, et al. Critical Choices: What Information Do Patients Want When Selecting a Hospital for Cancer Surgery? J Oncol Pract. 2018; 14(8): e505–e512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Marang-van de Mheen PJ, Dijs-Elsinga J, Otten W, et al. The relative importance of quality of care information when choosing a hospital for surgical treatment: a hospital choice experiment. Med Decis Making. 2011; 31(6): 816–27. [DOI] [PubMed] [Google Scholar]
- 8.Lubalin JS and Harris-Kojetin LD. What do consumers want and need to know in making health care choices? Med Care Res Rev. 1999; 56 Suppl 1: 67–102; discussion 103-12. [DOI] [PubMed] [Google Scholar]
- 9.McGuckin M, Waterman R, and Shubin A. Consumer attitudes about health care-acquired infections and hand hygiene. Am J Med Qual. 2006; 21(5): 342–6. [DOI] [PubMed] [Google Scholar]
- 10.Cella D, Choi SW, Condon DM, et al. PROMIS((R)) Adult Health Profiles: Efficient Short-Form Measures of Seven Health Domains. Value Health. 2019; 22(5): 537–544. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Luborsky M, The Identification and Analysis of Themes and Patterns, in Qualitative Methods in Aging Research, Gubrium J and Sankar A, Editors. 1994, Sage: Thousand Oaks, CA. [Google Scholar]
- 12.Bradley EH, Curry LA, and Devers KJ. Qualitative Data Analysis for Health Services Research: Developing Taxonomy, Themes, and Theory. Health Serv Res. 2007; 42(4): 1758–1772. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Merkow RP, Ju MH, Chung JW, et al. Underlying reasons associated with hospital readmission following surgery in the United States. JAMA. 2015; 313(5): 483–95. [DOI] [PubMed] [Google Scholar]
- 14.Pevnick JM and Schnipper JL. Exploring How to Better Measure and Improve the Quality of Medication Reconciliation. Jt Comm J Qual Patient Saf. 2017; 43(5): 209–211. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Ziemba JB, Allaf ME, and Haldeman D. Consumer Preferences and Online Comparison Tools Used to Select a Surgeon. JAMA Surg. 2017; 152(4): 410–411. [DOI] [PubMed] [Google Scholar]
- 16.Schlesinger MJ, Rybowski L, Shaller D, et al. Americans' Growing Exposure To Clinician Quality Information: Insights And Implications. Health Aff (Millwood). 2019; 38(3): 374–382. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Singh K, Meyer SR, and Westfall JM. Consumer-Facing Data, Information, And Tools: Self-Management Of Health In The Digital Age. Health Aff (Millwood). 2019; 38(3): 352–358. [DOI] [PubMed] [Google Scholar]
- 18.Austin JM, Jha AK, Romano PS, et al. National hospital ratings systems share few common scores and may generate confusion instead of clarity. Health Aff (Millwood). 2015; 34(3): 423–30. [DOI] [PubMed] [Google Scholar]
- 19.Mukamel DB, Amin A, Weimer DL, et al. When Patients Customize Nursing Home Ratings, Choices And Rankings Differ From The Government's Version. Health Aff (Millwood). 2016; 35(4): 714–9. [DOI] [PubMed] [Google Scholar]
- 20.Doering N and Maarse H. The use of publicly available quality information when choosing a hospital or health-care provider: the role of the GP. Health Expect. 2015; 18(6): 2174–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Birk HO and Henriksen LO. Which factors decided general practitioners' choice of hospital on behalf of their patients in an area with free choice of public hospital? A questionnaire study. BMC Health Serv Res. 2012; 12: 126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Gusmano MK, Maschke KJ, and Solomon MZ. Patient-Centered Care, Yes; Patients As Consumers, No. Health Aff (Millwood). 2019; 38(3): 368–373. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.