Skip to main content
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2017 Sep 20.
Published in final edited form as: Health Psychol. 2010 Jul;29(4):454–459. doi: 10.1037/a0019881

Choice numeracy and physicians-in-training performance: the case of Medicare Part D

Yaniv Hanoch 1,*, Talya Miron-Shatz 2, Helen Cole 3, Mary Himmelstein 4, Alex D Federman 5
PMCID: PMC5605761  NIHMSID: NIHMS903508  PMID: 20658834

Abstract

In choosing a prescription plan, Medicare beneficiaries in the US usually face over 50 options. Many have turned to their physicians for help with this complex task. However, exactly how well do physicians navigate information on Part D plans is still an open question. In this study, we explored this unanswered question by examining the effect of choice-set size and numeracy levels on a physician-in-training’s ability to choose appropriate Medicare drug plans. Consistent with our hypotheses, increases in choice sets correlated significantly with fewer correct answers, and higher numeracy levels were associated with more correct answers. Hence, our data further highlight the role of numeracy in financial- and health-related decision making, and also raise concerns about physicians’ ability to help patients choose the optimal Part D plan.

Keywords: Choice, Decision Making, Medicare part D, numeracy

Introduction

The Medicare Prescription Drug, Improvement, and Modernization Act of 2006, the most significant health care policy change in the U.S. since the inception of Medicare, introduced a prescription drug coverage option (Medicare Part D) for Medicare beneficiaries. Despite its promise, the program has attracted much criticism, in part because of its complex design. Medicare beneficiaries must choose from over 50 drug plans in most Part D markets (Kaiser Family Foundation, 2007), as well as a dizzying array of plan features, including variable monthly premiums and drug co-pays, different drug formularies and formulary structures, and options for gap (or “donut hole”) coverage and mail order pharmacy. It should come as no surprise that a majority of older adults (65 and above) have judged the program too difficult to navigate (Kaiser Family Foundation, 2006).

To cope with such challenges, older adults can obtain information and assistance from family members, friends, Medicare (either by phone or via the website), outreach and support groups like state health insurance counseling and assistance programs (SHIPs), media campaigns, and healthcare providers. Among these resources, healthcare workers may be particularly important. As prescribers and confidants familiar with their patients' health, financial, and social situations, physicians are often sought out for guidance on prescription drug coverage decisions. Indeed, earlier studies on Medicare beneficiaries indicate a widespread expectation that physicians be well-informed about the program (Mallya & Bazmemore, 2006). Over 30% of beneficiaries would likely enlist their doctor’s assistance—second only to Medicare sources—in deciding on Medicare part D and physicians (Kaiser Family Foundation, 2005), and a majority believe their doctor would spend time helping them choose the best drug plan (Kaiser Family Foundation, 2005). Because physicians are often called upon to assist with Part D decisions, it is vital to examine whether they can accurately process the requisite numerical information and appropriately guide their patients.

Earlier research indicates that choice set size and numeracy skills are key factors affecting an individual's ability to wade through the complexities of health insurance options (Iyengar & Lepper, 2000; Iyengar, Wells, & Schwartz, 2006; Iyengar, Huberman, & Jiang, 2004; Hanoch, Rice, Cummings, & Wood 2009). Unfortunately, research on physicians’ numeracy levels is scant, even though numeracy is essential for practicing evidence-based medicine (Rao, 2008). What little research has been done suggests that physicians may have some difficulty interpreting probability information from mammography screens (Ancker & Kaufman, 2007; Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz, & Woloshin, 2008; Estrada, Barnes, Collins, & Byrd, 1999). These studies, however, do not indicate how well physicians interpret insurance information—a potentially less familiar topic than, say, mammography and breast cancer risk.

We therefore examined in this study whether physicians' numeracy levels affect their ability to evaluate health insurance plans and whether a larger number of plans erodes the capacity to choose appropriately. We predicted that the performance of physicians-in-training would decrease as a function of the number of drug plans presented and that numeracy levels would moderate the effect of a larger choice set.

Method

Participants

Medical students and internal medicine residents (40 3rd year, 27 4th year, and 33 medical residents) at a large teaching hospital in the Northeast were recruited and tested between November 2008 and March 2009 (N = 100). The mean age was 27.2 (range, 23 to 39), and 53% were male. There were no significant differences based on age, gender, numeracy, or level of medical training between randomly assigned experimental groups.

Procedure and materials

The study was approved by the appropriate institutional review board. The instruments and design of the present study are based on an earlier investigation (Hanoch et al., 2009). A member of the research team briefly described the study and distributed surveys, the completion of which required approximately 10 minutes.

Respondents were randomly given one of three surveys. The surveys shared the same core questions and differed only in the number of plans to be evaluated by each participant. These items, described in greater detail below, compared three, ten, or twenty Medicare plans (see Appendix 1). Thirty-four study respondents received the survey that compared three plans, 30 received the survey with 10 plans, and 36 received the survey that included 20 plans. Information about stand-alone Medicare prescription drug plans was obtained from the Medicare website (www.medicare.gov) and selected to represent a range of plan options.1 We substituted single letters for the names of Medicare prescription plans to avoid bias associated with name recognition.2 Respondents were presented with a table including the following information about each drug plan: total estimated annual cost (including both premiums and cost-sharing requirements), annual deductible, availability of drugs by mail order, the number of local pharmacies participating in the plan, and distance in miles from the respondent to the nearest pharmacy. After reviewing this information, respondents answered questions about what plan they would advise two hypothetical patients to choose, based on a brief summary of the relevant concerns of each patient.

The main outcome variable was comprehension, and the comprehension items were chosen to test respondents’ ability to interpret tabular information, an essential skill for evaluating information provided by Medicare. Four dichotomous comprehension variables indicated whether a respondent answered correctly or incorrectly identified the health plans that: (1) minimized the total annual cost of prescription drugs; (2) minimized annual costs so long as drugs could be obtained by mail order; (3) included the most pharmacies, and (4) did not offer mail-order prescriptions but included the closest pharmacy. A continuous variable, total comprehension, was then derived from responses to each dichotomous comprehension variable: respondents received 1 point for each correct answer (4 points maximum).

Finally, participants completed an 11-item numeracy scale (Lipkus, Samsa & Rimer, 2001) assessing basic understanding of probabilities, since such knowledge is required for health insurance decision-making. Each item on the scale was coded as correct or incorrect, and a score from 0 to 11 was assigned by tabulating correct responses.

Results

Numeracy

Numeracy scores ranged from 5 to 11 (M = 10.43, SD = 1.00), with 64% of our sample scoring 11. Therefore, using a median split of numeracy (Peters et al., 2006) would have been futile; we treated numeracy as a continuous variable instead.

Comprehension

Over two-thirds of the sample (68%) chose the appropriate Medicare prescription drug plan across all situations. Questions involving annual cost generated the most errors: 77% chose the correct plan that minimized total annual cost, and 86% chose the appropriate plan that both minimized annual cost and allowed mail-order prescriptions.

Between-group differences in comprehension based on number of plans

A detailed description of results from Fisher’s exact tests on comprehension are shown in Table 1. Respondents who compared either 10 or 20 plans had more trouble identifying the plan that allowed mail-order prescriptions and minimized the total annual cost (Fisher’s exact p <.00) than those who analyzed only three plans. Similarly, respondents who evaluated just three plans did better in choosing the plan that minimized total annual cost than those who dealt with 10 or 20, with the difference marginally significant (Fisher’s exaxt p <.10). Respondents assigned to only three plans also scored significantly higher on total comprehension than did respondents assigned to 10 or 20 plans: F (2, 99) = 4.69, p = .01.

Table 1.

Between group differences in comprehension based on number of plans

Selection of Plan that Achieves the
Following Objectives
3 plans 10 plans 20 plans Total Fisher’s
exact
Φ
1. Plan that minimizes total annual cost 0.10* .21*
  % choosing incorrectly 4 7 12 23
  % choosing correctly 30 23 24 77
  Total 34 30 36 N =100

2. Plan that provides access to the most pharmacies 0.33 .19
  % choosing incorrectly 0 0 2 2
  % choosing correctly 34 30 34 98
  Total 34 30 36 N =100

3. Plan that provides access to the closest pharmacy, without a mail order option 0.15 .21*
  % choosing incorrectly 1 5 2 8
  % choosing correctly 33 25 34 92
  Total 34 30 36 N =100

4. Minimize total annual cost, and allows mail-order prescriptions 0.00** .29**
  % choosing incorrectly 0 7 7 14
  % choosing correctly 34 23 29 86
  Total 34 30 36 N =100
*

p < .10.

**

p < .05.

Predictors of comprehension

Predicting total comprehension by number of plans, numeracy, class year, and age yielded a significant model (F (4, 93) = 34.43, P= .003) with an R2 = .17. Number of plans (B= −.28 (SE =.09), p=.00) and numeracy (B= .21, (SE= .08), p=.01) emerged as the only significant predictors. Neither level of training (3rd year medical student, 4th year medical student, or resident) nor age contributed significantly to the model (ΔR2=.02; F [2, 89] = .94, ns).

Discussion

Can physicians-in-training successfully navigate the new Medicare Part D program? In line with an earlier study (Hanoch et al., 2009), our results show that even physicians-in-training perform more poorly when choice size is larger. That is, even highly educated and numerate individuals find our tasks complicated. Indeed, respondents who compared 10 or 20 drug plans (as opposed to just three) were significantly less likely to identify the plan that minimized the total annual costs, or the plan that both minimized the annual costs and allowed mail order. These results remain significant when we examine the respondents’ ability to answer not just one of the questions but all four factual questions correctly. Our findings support earlier indications that most (92%) physicians find Medicare Part D too complicated (Kaiser Family Foundation, 2006). The program’s complexities make it difficult to identify which drugs are covered by plan formularies, so that physicians sometimes prescribe drugs that fall outside their patients’ formularies (Tseng, Mangione, Brook, Keeler, & Dudely 2007). While the American Academy of Family Physicians’ educational campaign has foreshadowed this possibility (Mallya & Bazemore, 2006), our findings raise further concerns over physicians’ ability to help older adults choose a drug plan.

These results also highlight the fact that physicians-in-training can occasionally fail simple data interpretation tasks, illustrating the important role of numeracy in evaluating insurance-related information.3 Our data, therefore, not only fit well with earlier studies’ findings, but also extend the field to a new domain. The results support the growing body of evidence showing that numeracy plays an important role in medical students’ ability to interpret medical data (Sheridan & Pignone, 2002), physicians’ capacity to practice evidence-based medicine (Rao, 2008), and clinicians’ ability to communicate risk information to patients (Gigerenzer et al., 2008; Nelson, Reyna, Fagerlin, Lipkus & Peters, 2008). Our data also corroborate earlier indications that a range of health care professionals (physicians, nurses, doctorate faculty, and medical students) have trouble answering numeracy questions (Estrada, Barnes, Collins & Byrd., 1999) and that higher numeracy levels correlate positively with correct interpretation of medical data (Sheridan & Pignone, 2002). Thus, we concur with previous studies arguing for the need to provide physicians with greater numeracy education. Finally, our results extend existing work into the domain of health insurance, demonstrating that numeracy levels are also associated with the ability to process health insurance information. These findings highlight the need for enhancing numeracy skills among medical students and physicians (Davis & Wolf, 2004).

Nonetheless, our study has a number of limitations. First of all, our sample is one of convenience and may not accurately represent the general physician population. Because our sample consisted of medical students and residents, it is possible that their knowledge and experience with Medicare do not reflect the ability of more experienced physicians. In addition, their decisions did not directly affect real patients, only imaginary ones. Thus, participants who had to review 10 or 20 insurance plans might have been less motivated to carefully review the information and hence made more mistakes. Would practicing physicians, who have more experience with Medicare and are willing to devote the necessary time and effort, be better at finding the best insurance plan for their patients? Further studies are needed to address this important question.

Since the odds of being correct by chance alone are higher when there are only 3 (vs. 10 or 20) insurance plans, our results could have been driven by chance rather than by the difficulty of the task. Regardless of the driving factor, Medicare beneficiaries would benefit from a simplified program and fewer number of plans (Cummings, Rice & Hanoch, 2009). Additionally, despite our efforts to provide a decision task that mimics the actual Medicare drug benefit, it was not possible to render them identical.

Limitations notwithstanding, we believe that our results are conservative by nature and that results obtained with practicing physicians would be similar. As noted earlier, the decision environment faced by respondents was far simpler than real world situations: there was a maximum of 20 plans, rather than the 50-plus Part D plans available on the market. Moreover, complex elements of Part D, such as drug formularies and “donut holes,” were not part of the study. Perhaps most importantly of all, respondents did not have to navigate through the Medicare website and input detailed information about their patients’ prescriptions. Rather, our simplified version of the tasks is likely to have mitigated the real world complexities. These difficulties are probably best exemplified by findings highlighting practicing physicians’ troubles in prescribing drugs that appear on their patients’ plan formularies (Tseng, Mangione, Brook, Keeler, & Dudely, 2007).

A growing number of studies have come to challenge the merit of offering more choice, arguing that a greater choice set increases task complexity. Our findings support this notion and, thus, raise concerns about the capacity of physicians-in-training to successfully navigate Medicare Part D and help their patients pick the best drug plan. Given the financial and medical ramifications involved, policymakers must recognize these difficulties and their potential harms.

Figure 1.

Figure 1

Comprehension by number of plans reviewed.

Acknowledgments

This work was supported by a Robert Wood Johnson Foundation Investigator Award in Health Policy Research. Dr. Federman received support from a Paul B. Beeson career development award in aging related research from the National Institute on Aging. The views expressed imply no endorsement by the Robert Wood Johnson Foundation. We thank Thomas Rice and 3 anonymous reviewers for comments on earlier drafts.

Appendix 1

We start by presenting you with a hypothetical situation. Please read the following carefully and answer the questions that follow using only the information you gained by reading the example.

Imagine that one of your friends, whom we’ll call Bill, has asked you to help him in choosing a Medicare prescription drug plan. He has made it clear that he is only concerned about saving money, and therefore would like to enroll in the plan that minimizes his total combined annual cost for prescription drugs and the insurance for those drugs. We will provide you with a card containing information about several insurance companies covering prescription drugs. After you review this information in Card A, you will be asked several questions. Please choose the answer which you think is the best one.

We will also provide you with a card containing the definitions of various terms used throughout this experiment that can help you in answering the questions.

If you have any questions during the study, do not hesitate to ask a member of the project team.

Please answer the following questions using Card A:

  1. Given Bill’s desire to minimize total annual cost of prescription drugs, which one of the drug plans listed on Card A would you recommend that he choose? Please write down the Company Letter.
    • Company Letter _____________
  2. Looking at the information in Card A, which company has the largest number of pharmacies? Please write down the Company Letter.
    • Company Letter _____________
  3. Which of the companies listed does NOT offer mail order but has the closest pharmacy location? Please write down the Company Letter.
    • Company Letter _____________

CARD A

Company
Letter
Total
combined
annual cost
Mail
order
Number
of
pharmacies
Distance to
closest
pharmacy (in
miles)
Annual
deductible
Monthly
cost share
A $4,860 Yes 14 3 $0 $140
B $5,230 No 13 6 $0 $160
C $4,530 Yes 20 4 $250 $250
D $7,020 No 16 10 $250 $420
E $5,140 Yes 20 8 $100 $210
F $4,890 No 17 5 $0 $220
G $4,830 No 16 9 $250 $270
H $6,740 Yes 14 8 $0 $380
I $4,870 Yes 18 2 $0 $200
J $6,250 Yes 18 1 $0 $310
K $4,410 No 12 2 $250 $240
L $4,880 No 15 9 $250 $130
M $4,950 No 15 6 $0 $230
N $6,870 Yes 19 7 $100 $310
O $4,990 No 21 8 $100 $210
P $5,250 Yes 16 1 $100 $220
Q $4,950 Yes 12 3 $0 $230
R $4,800 No 16 6 $50 $170
S $4,830 No 13 15 $0 $180
T $6,720 Yes 15 7 $0 $420

Footnotes

1

To increase the ecological validity of the study material, we completed a drug selection process on the official Medicare website (using the general search option). After inputting general information (e.g., zip code and age), we entered the names of two commonly prescribed medications (Lipitor and Nexium), which provided us with a list of about 50 drug plans. From the list, we picked 20 with differing total combined annual costs (from the cheapest to the most expensive). Next, to reduce the number of plans to 10, we chose a subset from the 20 plans that offered the lowest total combined annual costs. A similar procedure was used to reduce the number of plans from 10 to 3. All individuals who evaluated 20 (10 and 3) drug plans evaluate the same drug plan information. We had decided to use annual costs because price is probably important for selection of drug plans. Moreover, given the complexity of the Medicare Part D program, it is impossible to capture the entire spectrum of possible plans or even to characterize the typical plan. For example, a drug plan that offers the lowest total combined annual cost for certain medications might actually be the most expensive one if other medications are required; some drug plans offer coverage during the coverage gap and some plans don’t; some offer such coverage only for generic medication and some for both generic and brand names; certain plans charge no monthly premiums while others do; etc.

2

Our decision to use letters instead of company names followed earlier experiments that examined participants’ understanding of health care information (Hibbard, Slovic, Peters, Finucane, & Tusler, 2001; Finucane, Mertz, Slovic, & Schmidt, 2002). That is, our study aimed to investigate participants’ abilities to compare drug plan information (e.g., identify the drug plan that offers the lowest total annual cost) in relationship to choice size. Thus, while we acknowledge that company name can be an important factor in people’s decision, we wished to keep our data free of brand familiarity, as it would not have permitted use of objective measures to evaluate participants’ errors rate. For example, if a participant chose based on company name (e.g., Blue Cross) a drug plan that does not offer the lowest combined annual cost, it would have been difficult to determine whether the decision represented an error or simply reflected preference or familiarity. Furthermore, insurance companies might each offer more than one drug plan to choose from. For instance, Blue Cross’s plans may differ on deductible values, coverage during the gap, extent of coverage on brand name drugs, etc. Thus, choosing a drug plan based on company name might reduce but not eliminate the complexity of the task. At the same time, we are fully aware that doctors and Medicare beneficiaries may use company name as a cue for drug plan decisions. Evaluating this possibility, however, would require a different study design, one that is beyond the scope of our study. Our current design, then, may have created a more complicated choice environment, since it did not allow participants to rely on company name.

3

Because we did not include certain other measures (e.g., processing speed and working memory), it is difficult to conclude whether numeracy alone can explain our results–other factors could have contributed to our findings.

Contributor Information

Yaniv Hanoch, University of Plymouth.

Talya Miron-Shatz, Princeton University.

Helen Cole, Mount Sinai School of Medicine.

Mary Himmelstein, Princeton University.

Alex D. Federman, Mount Sinai School of Medicine

References

  1. Ancker JS, Kaufman D. Rethinking health numeracy: A multidisciplinary literature review. Journal of the American Medical Informatics Association. 2007;14:713–721. doi: 10.1197/jamia.M2464. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Cummings JR, Rice T, Hanoch Y. “Who Thinks That Part D Is Too Complicated? Survey Results on the Medicare Prescription Drug Benefit”. Medical Care Research and Review. 2009;66(1):97–115. doi: 10.1177/1077558708324340. [DOI] [PubMed] [Google Scholar]
  3. Davis TC, Wolf MS. Health literacy: Implications for family medicine. Family Medicine. 2004;36:595–598. [PubMed] [Google Scholar]
  4. Estrada C, Barnes V, Collins C, Byrd JC. Health literacy and numeracy. Journal of the American Medical Association. 1999;11:527. doi: 10.1001/jama.282.6.527. [DOI] [PubMed] [Google Scholar]
  5. Finucane ML, Mertz CK, Slovic P, Schmidt ES. Task complexity and older adults' decision-making competence. Journal of Psychology and Aging. 2005;20:71–84. doi: 10.1037/0882-7974.20.1.71. [DOI] [PubMed] [Google Scholar]
  6. Gigerenzer G, Gaissmaier W, Kurz-Milcke E, Schwartz LM, Woloshin S. Helping doctors and patients make sense of health statistics. Psychological Science in the Public Interest. 2008;8:53–96. doi: 10.1111/j.1539-6053.2008.00033.x. [DOI] [PubMed] [Google Scholar]
  7. Hanoch Y, Rice T, Cummings J, Wood S. How Much Choice is Too Much?: The Case of the Medicare Prescription Drug Benefit. Health Service Research. 2009;44:1157–1168. doi: 10.1111/j.1475-6773.2009.00981.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Hibbard JH, Slovic P, Peters E, Finucane ML, Tusler M. Is the informed-choice policy approach appropriate for Medicare beneficiaries? Health Affairs. 2001;20:199–203. doi: 10.1377/hlthaff.20.3.199. [DOI] [PubMed] [Google Scholar]
  9. Iyengar S, Huberman G, Jiang W. How much choice is too much? Contributions to 401(k) retirement plans. In: Mitchell OS, Utkus S, editors. Pension Design and Structure: New Lessons from Behavioral Finance. Oxford: Oxford University Press; 2004. pp. 83–95. [Google Scholar]
  10. Iyengar SS, Lepper MR. When choice is demotivating: Can one desire too much of a good thing. Journal of Personality and Social Psychology. 2000;76:995–1006. doi: 10.1037//0022-3514.79.6.995. [DOI] [PubMed] [Google Scholar]
  11. Iyengar SS, Wells RE, Schwartz B. Doing better but feeling worse: Looking for the "best" job undermines satisfaction. Psychological Science. 2006;17:143–150. doi: 10.1111/j.1467-9280.2006.01677.x. [DOI] [PubMed] [Google Scholar]
  12. Kaiser Family Foundation. [Retrieved May 10, 2007];The Medicare Drug Benefit: Beneficiary Perspectives Just Before Implementation. 2005 from http://www.kff.org/kaiserpolls/upload/The-Medicare-Drug-Benefit-Beneficiary-Perspectives-Just-Before-Implementation-Chartpack.pdf.
  13. Kaiser Family Foundation. [Retrieved February 18, 2008];National Survey of Pharmacists and National Survey of Physician. 2006 from http://www.kff.org/kaiserpolls/upload/7556.pdf.
  14. Kaiser Family Foundation. [Retrieved February 18, 2008];Medicare Part D Plan Characteristics, By State, 2008: Fact Sheet. 2007 from www.kff.org/medicare/upload/7426_04.pdf.
  15. Lipkus IM, Samsa G, Rimer BK. General performance on a numeracy scale among highly educated samples. Medical Decision Making. 2001;21:37–44. doi: 10.1177/0272989X0102100105. [DOI] [PubMed] [Google Scholar]
  16. Mallya G, Bazemore A. Medicare Part D: practical and policy implications for family physicians. American Family Physician. 2006;73:395–396. [PubMed] [Google Scholar]
  17. Nelson W, Reyna VF, Fagerlin A, Lipkus I, Peters E. Clinical implications of numeracy: Theory and practice. Annals of behavioral medicine. 2008;35:261–274. doi: 10.1007/s12160-008-9037-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Peters E, Vastfjall D, Slovic P, Mertz CK, Mazzocco K, Dickert S. Numeracy and decision making. Psychological Science. 2006;17:407–413. doi: 10.1111/j.1467-9280.2006.01720.x. [DOI] [PubMed] [Google Scholar]
  19. Rao G. Physician numeracy: Essential skills for practicing evidence–based medicine. Family Medicine. 2008;40:354–358. [PubMed] [Google Scholar]
  20. Schwartz B. The paradox of choice: Why more is less. New York: Harper Perennial; 2005. [Google Scholar]
  21. Sheridan SL, Pignone M. Numeracy and the medical student’s ability to interpret data. Effective Clinical Practice. 2002;5:35–40. [PubMed] [Google Scholar]
  22. Tseng CW, Mangione CM, Brook RB, Keeler E, Dudely A. Identifying widely covered drugs and drug coverage variation among Medicare Part D formularies. Journal of the American Medical Association. 2007;297:2596–2602. doi: 10.1001/jama.297.23.2596. [DOI] [PubMed] [Google Scholar]

RESOURCES