Abstract
BACKGROUND
Physicians’ performance on measures of clinical quality is rarely available to patients. Instead, patients are encouraged to select physicians on the basis of characteristics such as education, board certification, and malpractice history. In a large sample of Massachusetts physicians, we examined the relationship between physician characteristics and performance on a broad range of quality measures.
METHODS
We calculated overall performance scores on 124 quality measures from RAND’s Quality Assessment Tools for each of 10,408 Massachusetts physicians using claims generated by 1.13 million adult patients. The patients were continuously enrolled in 1 of 4 Massachusetts commercial health plans during 2004–2005. Physician characteristics were obtained from the Massachusetts Board of Registration in Medicine. Associations between physician characteristics and overall performance scores were assessed using multivariate linear regression.
RESULTS
The mean overall performance score was 62.5%% (5th to 95th percentile range, 48.2% to 74.9%). Three physician characteristics were independently associated with significantly higher overall performance: female gender (1.6 percentage points higher than male, p<0.001), board certification (3.3 percentage points higher than non-certified, p<0.001), and graduation from a domestic medical school (1.0 percentage points higher than international, p<0.001). There was no association between performance and malpractice claims or disciplinary action.
CONCLUSION
Few characteristics of individual physicians were associated with higher performance on measures of quality, and observed associations were small in magnitude. Publicly available characteristics of individual physicians are poor proxies for performance on clinical quality measures.
INTRODUCTION
To improve the quality of care received by their beneficiaries, some health plans use physician report cards and tiered physician networks to steer their members towards physicians who provide high-quality care. However, most patients do not have access to physician quality measures. Further, the quality metrics available to some patients are limited in scope and reflect only a few aspects of overall quality of care. Patients are therefore encouraged to use publicly available proxies for clinical performance when choosing a physician. Independent organizations such as the AARP advise patients to choose physicians based on characteristics such as education, disciplinary action, and board certification status.1 The consumer website HealthGrades limits its “recognized doctor” and “five-star doctor” labels to physicians who are board certified, who have never had their license revoked, and who are free of disciplinary actions or malpractice claims.2 Malpractice claims and board certification status, along with procedure-specific experience, are judged by consumers to be much more indicative of the quality of care delivered by a physician than ratings by government agencies or independent medical institutions.3
There appears to be a tacit belief that these physician characteristics are a signal for clinical quality. However, the value of publicly available individual physician characteristics as predictors of clinical quality is unclear. In previous studies that examine the relationship between individual physician characteristics and quality of care few definitive or broadly applicable conclusions have emerged. The relationship between performance on quality measures and physicians’ history of malpractice claims or disciplinary actions has not been studied to our knowledge.4–6 In general, studies have found an inverse relationship between years of experience and performance on quality measures.7–9 There have been mixed findings in the relationship between quality and other characteristics such as gender,8–14 board certification status,8, 15, 16 medical school site (i.e. international vs. domestic).8, 17, 18 Previous investigations of relationships between individual physician characteristics and performance on quality measures have been limited by the number of physicians assessed, the available physician characteristics, and the scope and validity of quality metrics used. Much of the previous literature related to physician characteristics and clinical quality has had a narrow clinical focus, each study examining only a limited range of processes, conditions, or specialties.
In this study we examined, in a large sample of Massachusetts physicians, the relationship between a number of physician characteristics and performance on a broad range of quality measures.
METHODS
PATIENT SAMPLE
Physician performance scores were created using a de-identified aggregated claims dataset of 1.13 million patients between the ages of 18 and 65 who were enrolled continuously in 1 of 4 Massachusetts commercial health plans in 2004–5. Taken together, the 4 plans constituted over 85% of the commercial market in the state. The dataset included all professional, inpatient, facility, and pharmacy claims. Physicians were linked across the 4 health plans using a crosswalk developed by the Massachusetts Health Quality Partners (MHQP) that connects a unique physician identifier to the provider numbers used by each health plan.19 Children younger than 18 years of age were excluded because no pediatric quality measures were used. Elders (>65) were also excluded because co-insurance with Medicare was inconsistently recorded, and the plans could not reliably identify those for whom Medicare was the primary payer.
PHYSICIAN SAMPLE
The MHQP maintains a database of all providers who have a contract with any of the major commercial health plans in the state. From this sample of providers, we eliminated those who practiced outside Massachusetts and those who did not bill at least one claim to any of the 4 health plans in 2004–5. We then eliminated non-physicians (i.e. podiatrists, chiropractors, acupuncturists), physicians with no assigned specialty, pediatricians, and specialties with no applicable quality measures or direct patient care (e.g., pathology, radiology). After these exclusions, physicians in 23 specialties contributed data to the analysis.
DATA ON PHYSICIAN CHARACTERISTICS
Publicly available data on individual physician characteristics was obtained from the Massachusetts Board of Registration in Medicine (http://profiles.massmedboard.org/MA-Physician-Profile-Find-Doctor.asp). The Board publicly releases, for each physician, information on birth date, medical school graduation date, medical school attended, board certification status, gender, payment on malpractice claims, and disciplinary actions. These data are entered and updated by physicians at the time of licensure and re-licensure. However, malpractice and disciplinary information are maintained by the Board and are not self-entered by physicians. From this database we eliminated physicians with a limited license (i.e. residents). Experience was measured by years since medical school graduation. Medical schools in the United States were matched to their 2008 U.S. News and World Report’s rankings in research and primary care.20 Malpractice claims included those on which a payment was made between March 30, 1998 and February 28, 2008. The Board’s disciplinary archives listed all disciplinary and public actions by the board since June 9, 1999 through June 18, 2008.21 We did not include 5 publicly available variables for analysis. Two of these variables (criminal convictions, hospital disciplinary actions) were very rare among physicians. Two variables (publications published, awards) were inconsistently entered by physicians, and one variable (work site) had unclear definitions. For example, it was unclear how a physician might choose between “educational institution”, “hospital”, or “clinic”.
MEASURING QUALITY OF CARE
We used the RAND claims-based Quality Assessment (QA) tools to assess performance on measures of clinical quality. The development of the QA tools measures has been described in previous publications.22,23 Briefly, RAND staff selected conditions that were identified as leading causes of death, illness, and utilization of healthcare; staff physicians reviewed established national guidelines and medical literature to identify key processes of care subject to potential overuse and underuse throughout the continuum of care for each condition. Four nine-member multispecialty expert panels, each with a diversity of geography, practice setting, and sex, were convened to assess the validity of the indicators using the RAND–UCLA modified Delphi method. The QA Tools measures were initially developed to be abstracted from medical records and included 439 measures; these have been subsequently adapted to be scored using claims records. The claims-based QA Tools measures used in our analyses include 124 indicators of quality of care for 22 acute and chronic conditions, as well as preventive care which are listed in the appendix.
Instances when recommended care was indicated or provided were attributed to the individual physicians who triggered the indicator. Each physician’s composite performance score was created by dividing the number of instances in which recommended care was delivered by the number of instances patients were eligible for such care and that were assigned to that physician. This composite method has been described as the “overall score” method in previous literature.24 In order to prevent differences in the ease of delivering needed care (e.g., the mean rate of mammography for the state is much higher than the mean rate of cervical cancer screening) from affecting physicians’ overall performance scores, we standardized the expected performance on each indicator by subtracting its statewide mean from each physician’s score on that indicator. This process created a “measurement difficulty-adjusted” performance composite score whose mean was 0 across all physicians.25
DATA ANALYSIS
We created multivariate linear regression models to examine the associations between physician characteristics and performance scores. The unit of analysis was the individual physician. The dependent variable was the composite difficulty-adjusted performance score. The independent variables were physician gender, board certification status, experience (years since graduation from medical school), medical school location (domestic or international), medical school ranking (within or below the top 10 in the 2008 U.S News rankings), malpractice claims (none vs. one or more in the last 10 years), and disciplinary actions by the board (none vs. one or more in the last 10 years). The regression was weighted by the number of quality measure opportunities attributed to each physician.
We ran several different versions of the regression model using different subsets of physicians and performance data: (1) all physicians and all indicators; (2) all physicians, but with separate regressions for acute, chronic, and preventive care indicators; (3) all physicians, but with separate regressions for female-patient-specific and male-patient-specific indicators (e.g., recommended prenatal or mammography care for women, and recommended benign prostatic hypertrophy or sexually transmitted infection care for men); and (4) all indicators, but with separate regression models for the 5 specialties that averaged greater than 150 quality measure opportunities per physician (internal medicine, family/general practice, cardiology, obstetrics and gynecology, and endocrinology).
Performance scores are presented as the mean score for the group of physicians possessing each characteristic. We created these scores by solving the regression model created for each care-type or physician specialty to find the percentage-point difference in difficulty-adjusted performance score attributable to that characteristic. We then added that quantity to the unadjusted mean performance score to arrive at a quantity representing the percentage of recommended care that physicians with that characteristic provide, adjusted for the degree of difficulty of each measure. To address the testing of multiple comparisons, we calculated the critical P value that limited the false discovery rate (the expected rate of type 1 error among all significant statistical tests) to 5%.26 P values below this threshold were considered statistically significant. All statistical analyses were performed using SAS version 9.2 (SAS Institute, Inc., Cary, North Carolina).
RESULTS
Of the 30,122 physicians in the MHQP database, there were 12,959 physicians in the 23 selected specialties who had a full license, who practiced in Massachusetts, and who submitted one or more claims in 2004–5. We then excluded the 2,239 physicians with no attributed quality measures and the 302 physicians that could not be linked to the physician characteristics dataset. The remaining 10,408 (80.3%) physicians were the basis of our analysis. There were 1,704,686 quality measure opportunities included in the analysis, a mean of 163.8 events per physician (range, 1–3329).
The majority of physicians were male (70.1%), board certified (92.8%), domestically trained (83.0%), and in possession of allopathic medical degrees (97.7%) [Table 1]. They spanned a wide breadth of experience in practice; 15.2% had less than 10 years and 24.7% had 30 or more years of experience. Few made payments on malpractice claims in the last decade (10.2%), and fewer had disciplinary actions against them in that time (1.0%). Approximately 1 in 10 attended schools ranked in the top 10 by US News and World Report for research (12.6%) or primary care (9.8%) [Table 1]. The physicians were distributed across the 23 specialties, but 34.5% of the physicians in the sample practiced internal medicine [Table 2].
Table 1.
Characteristics of Physician Sample
| Characteristic | n | % |
|---|---|---|
| Degree | ||
| D.O. | 242 | 2.3% |
| M.D. | 10166 | 97.7% |
| Gender | ||
| Male | 7300 | 70.1% |
| Female | 3108 | 29.9% |
| Board Certification | ||
| None | 751 | 7.2% |
| ABMS or AOA | 9657 | 92.8% |
| Years in Practice | ||
| <10 Years | 1578 | 15.2% |
| 10–19 Years | 3224 | 31.0% |
| 20–29 Years | 3038 | 29.2% |
| 30–39 Years | 1832 | 17.6% |
| 40–49 Years | 637 | 6.1% |
| ≥ 50 Years | 99 | 1.0% |
| Medical School | ||
| International | 1767 | 17.0% |
| Domestic | 8641 | 83.0% |
| Malpractice | ||
| No Malpractice Claims | 9350 | 89.8% |
| 1+ Malpractice Claim | 1058 | 10.2% |
| Disciplinary Action | ||
| No Disciplinary Actions | 10307 | 99.0% |
| 1+ Disciplinary Actions | 101 | 1.0% |
| US News Research Rank | ||
| Attended a lower or unranked school | 9095 | 87.4% |
| Attended a top 10 school | 1313 | 12.6% |
| US News Primary Care Rank | ||
| Attended a lower or unranked school | 9387 | 90.2% |
| Attended a top 10 school | 1021 | 9.8% |
Table 2.
Specialty Distribution in Physician Sample
| Specialty | n | % |
|---|---|---|
| All Specialties | 10408 | |
| Allergy and Immunology | 79 | 0.8% |
| Cardiology | 482 | 4.6% |
| Cardiothoracic Surgery | 81 | 0.8% |
| Emergency Medicine | 560 | 5.4% |
| Endocrinology | 152 | 1.5% |
| Family/General Practice | 989 | 9.5% |
| Gastroenterology | 246 | 2.4% |
| General Surgery | 465 | 4.5% |
| Hematology/Oncology | 197 | 1.9% |
| Internal Medicine | 3587 | 34.5% |
| Nephrology | 102 | 1.0% |
| Neurological Surgery | 89 | 0.9% |
| Neurology | 375 | 3.6% |
| Obstetrics and Gynecology | 877 | 8.4% |
| Ophthalmology | 376 | 3.6% |
| Orthopedic Surgery | 491 | 4.7% |
| Otolaryngology | 178 | 1.7% |
| Psychiatry | 489 | 4.7% |
| Pulmonary & Critical Care | 206 | 2.0% |
| Radiation Oncology | 41 | 0.4% |
| Rheumatology | 90 | 0.9% |
| Urology | 203 | 2.0% |
| Vascular Surgery | 53 | 0.5% |
OVERALL PERFORMANCE
Among all physicians the mean unadjusted overall performance score was 62.5%, with an 5th to 95th percentile range of 48.2% to 74.9%. Performance scores varied by condition, ranging from 30.8% for cataract care to 68.0% for congestive heart failure care. Unadjusted performance scores for all physicians on the 20 most frequent occurring indicators are shown in Table 3.
Table 3.
Unadjusted Performance Scores for 20 Most Frequently Triggered Quality Measures
| Quality Measure | N, Eligible Events | Mean Score for Indicator | Max | Min |
|---|---|---|---|---|
| Women age 18 and older should have a Pap smear every three years. | 328,703 | 80.8% | 1 | 0 |
| All average risk adults age 50 to 80 should receive at least one of the following colon cancer screening tests: FOBT (if not done in the past 2 years), sigmoidoscopy (if not done in the past 5 years), colonoscopy (if not done in the past 10 years), double contrast barium enema (if not done in the past 5 years). | 211,508 | 56.7% | 1 | 0 |
| Patients with diabetes should have a glycosylated hemoglobin test every 6 months. | 101,408 | 64.8% | 1 | 0 |
| Patients with diabetes should have a follow-up visit at least every 6 months. | 101,404 | 63.0% | 1 | 0 |
| Patients should not be taking any of the following medications for treatment of acute low back pain: dexamethasone, other oral steroids, colchicine, anti-depressants. | 58,351 | 93.2% | 1 | 0 |
| Patients in whom pharmacological therapy for hyperlipidemia has been initiated should have their total cholesterol, HDL, and LDL rechecked within 4 months. | 51,446 | 49.1% | 1 | 0 |
| Treatment with anti-microbials for uncomplicated lower tract infections in women under age 65 should not exceed 7 days. | 47,035 | 44.9% | 1 | 0 |
| Patients with diabetes should have total serum cholesterol and HDL cholesterol tests documented. | 42,248 | 91.9% | 1 | 0 |
| Skull x-rays should not be part of an evaluation for headache. | 41,993 | 99.7% | 1 | 0 |
| Patients on warfarin should have an INR checked a minimum of every three months. | 41,977 | 47.2% | 1 | 0 |
| Patients under age 75 with preexisting heart disease who are not on pharmacological therapy for hyperlipidemia should have total cholesterol, HDL, and LDL level tested at least every 5 years. | 40,552 | 92.7% | 1 | 0 |
| Patients with new onset headache that is severe should receive an imaging study (CT, MRI). | 37,763 | 25.1% | 1 | 0 |
| Treatment for bacterial vaginosis should be with metronidazaole (orally or vaginally) or clindamycin (orally or vaginally) at the time of diagnosis. | 29,844 | 20.3% | 1 | 0 |
| Patients with diabetes should have an annual eye and visual exam. | 29,300 | 36.1% | 1 | 0 |
| Patients receiving pharmacological therapy for hyperlipidemia who have had a dosage or medication change should have total cholesterol, HDL, and LDL rechecked within 4 months of the change. | 28,739 | 42.8% | 1 | 0 |
| Patients with diabetes should have an annual measurement of urine protein (annual) documented. | 23,158 | 49.1% | 1 | 0 |
| Patients without preexisting coronary disease who are started on pharmacological treatment for hyperlipidemia should have had at least 2 measurements of their cholesterol (total or LDL) in the year before the start of pharmacological treatment. | 16,910 | 42.1% | 1 | 0 |
| Persons treated for pneumonia should have follow-up contact with a provider within 6 weeks after discharge or diagnosis. | 13,088 | 70.3% | 1 | 0 |
| Anti-anxiety agents should not be prescribed as a sole agent for the treatment of depression. | 11,727 | 61.6% | 1 | 0 |
| The first prenatal visit should occur in the first trimester. | 11,240 | 85.7% | 1 | 0 |
PHYSICIAN CHARACTERISTICS AND PERFORMANCE
In a multivariate model including all physicians and all types of care, female physician scored higher than male physicians (1.6 percentage points, p<0.001), board certified physicians scored higher than those without board certification (3.3 percentage points, p<0.001), and domestically trained physicians scored higher than internationally trained physicians (1.0 percentage point, p<0.001) [Table 4]. There were no statistically significant associations between performance and allopathic vs. osteopathic degree, medical school rankings, disciplinary actions, malpractice claims, or years of experience.
Table 4.
Associations between Physician Characteristics and Performance on Quality Measures by Type of Care Provided
| Type of Care | Gender Specific Care | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| All Measures | Acute | Chronic | Preventive | Male-Specific | Female-Specific | |||||||
| n, Physicians | 10,408 | 8,221 | 8,466 | 5,779 | 697 | 5,717 | ||||||
| n, Measure Opportunities | 1,704,686 | 448,961 | 706,238 | 546,425 | 2,019 | 524,234 | ||||||
| Mean, Quality Measure Opportunities per Physician | 163.8 | 54.6 | 83.4 | 94.6 | 2.9 | 91.7 | ||||||
| Parameter | Mean | p-value | Mean | p-value | Mean | p-value | Mean | p-value | Mean | p-value | Mean | p-value |
| Degree | ||||||||||||
| D.O. | 61.1% | 0.03 | 55.3% | 0.99 | 60.1% | 0.37 | 67.5% | 0.02 | 76.8% | 0.003† | 64.5% | 0.18 |
| M.D. | 62.5% | 55.3% | 60.8% | 70.3% | 60.8% | 65.7% | ||||||
| Gender | ||||||||||||
| Male | 61.9% | <.001† | 55.3% | 0.73 | 61.0% | 0.07 | 68.2% | <.001† | 60.6% | 0.22 | 64.0% | <.001† |
| Female | 63.6% | 55.4% | 60.4% | 73.6% | 65.8% | 68.4% | ||||||
| Board Certification | ||||||||||||
| None | 59.3% | <.001† | 53.6% | 0.001† | 58.5% | 0.04 | 64.7% | <.001† | 59.5% | 0.73 | 60.9% | <.001† |
| ABMS or AOA | 62.7% | 55.5% | 60.9% | 70.6% | 61.7% | 66.0% | ||||||
| Years in Practice | ||||||||||||
| <10 Years | 62.3% | 0.38 | 56.7% | <.001† | 60.8% | 0.88 | 68.3% | <.001† | 66.9% | 0.02 | 65.8% | 0.54 |
| 10–19 Years | 62.4% | 55.9% | 60.8% | 69.4% | 63.9% | 65.7% | ||||||
| 20–29 Years | 62.5% | 55.1% | 60.8% | 70.5% | 60.8% | 65.6% | ||||||
| 30–39 Years | 62.7% | 53.5% | 60.7% | 72.8% | 54.8% | 65.4% | ||||||
| 40–49 Years | 62.8% | 52.7% | 60.7% | 73.9% | 51.7% | 65.2% | ||||||
| ≥ 50 Years | 62.9% | 51.9% | 60.7% | 75.0% | 48.7% | 65.1% | ||||||
| Medical School | ||||||||||||
| International | 61.6% | <.001† | 55.1% | 0.32 | 60.6% | 0.57 | 68.0% | <.001† | 55.0% | 0.02 | 64.6% | 0.007† |
| Domestic | 62.6% | 55.4% | 60.8% | 70.7% | 63.2% | 65.9% | ||||||
| Malpractice | ||||||||||||
| No Malpractice Claims | 62.4% | 0.26 | 55.3% | 0.62 | 61.0% | <.001† | 69.8% | <.001† | 61.0% | 0.22 | 65.2% | <.001† |
| 1+ Malpractice Claim | 62.8% | 55.5% | 58.7% | 73.5% | 64.8% | 68.7% | ||||||
| Disciplinary Action | ||||||||||||
| No Disciplinary Actions | 62.5% | 0.02 | 55.4% | 0.15 | 60.8% | 0.22 | 70.2% | 0.15 | 61.6% | 0.20 | 65.7% | 0.18 |
| 1+ Disciplinary Actions | 60.0% | 53.4% | 59.2% | 66.4% | 53.3% | 62.7% | ||||||
| US News Research Rank | ||||||||||||
| Attended a lower or unranked school | 62.5% | 0.14 | 55.5% | 0.04 | 60.8% | 0.42 | 70.3% | 0.40 | 61.4% | 0.80 | 65.8% | 0.011† |
| Attended a top 10 school | 61.9% | 54.4% | 60.5% | 69.4% | 62.4% | 64.0% | ||||||
| US News Primary Care Rank | ||||||||||||
| Attended a lower or unranked school | 62.5% | 0.54 | 55.4% | 0.60 | 60.7% | 0.38 | 70.3% | 0.44 | 61.7% | 0.61 | 65.6% | 0.91 |
| Attended a top 10 school | 62.2% | 55.1% | 61.2% | 69.4% | 59.4% | 65.5% | ||||||
Statistically significant difference, allowing a maximum false discovery rate of 5% (critical P value = 0.0116).
The available physician characteristics only explained 2.8% of overall variation in physician performance. Separate regressions models for acute, chronic, and preventive care demonstrated that board certification was associated with higher quality on 2 of the 3 types of care (1.8 percentage points for acute care, p=0.001; 5.9 percentage points for preventive care, p<0.001) [Table 4]. Of the physician characteristics, the greatest differences in quality were generally seen among the preventive care measures (female 5.3 percentage points higher than male, p<0.001; board certified 5.9 percentage points higher than non-certified, p<0.001; domestically trained 2.7 percentage points higher than internationally trained, p<0.001, paying a malpractice claim 3.7 percentage points higher than vs. no paid malpractice claim, p<0.001).
Utilizing separate regression models for male and female-specific measures, we found that female physicians had significantly higher performance scores than male physicians on female-specific measures (4.4 percentage points higher, p<0.001) and male-specific measures (5.2 percentage points higher, p=0.22). The latter difference was not statistically significant [Table 4].
Using separate regression models for each of 5 common specialties in our physician population, we found no physician characteristics that were consistently associated with higher clinical quality across all specialties [Table 5]. However, the associations seen at the overall for all physicians and for all types of care paralleled those seen in internal medicine.
Table 5.
Associations between Physician Characteristics and Performance on Quality Measures by Physcician Specialty
| Cardiology | Endocrinology | Family/General Practice | Internal Medicine | Obstetrics and Gynecology | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| n, Physicians | 482 | 152 | 989 | 3,587 | 877 | |||||
| n, Quality Measure Opportunities | 98,202 | 22,805 | 277,555 | 953,053 | 248,451 | |||||
| Mean Number of Quality Measure Opportunites per Physician | 203.7 | 150.0 | 280.6 | 265.7 | 283.3 | |||||
| Parameter | Mean | p-value | Mean p-value | Mean | p-value | Mean | p-value | Mean | p-value | |
| Degree | ||||||||||
| D.O. | 66.2% | 0.78 | - | 0.99 | 61.5% | 0.30 | 61.7% | 0.17 | 63.3% | 0.05 |
| M.D. | 66.7% | 66.6% | 62.6% | 62.8% | 61.8% | |||||
| Gender | ||||||||||
| Male | 66.8% | 0.43 | 68.4% | 0.05 | 61.6% | <.001† | 62.0% | <.001† | 61.5% | 0.25 |
| Female | 66.2% | 64.5% | 63.8% | 64.1% | 62.2% | |||||
| Board Certification | ||||||||||
| None | 66.9% | 0.93 | 68.7% | 0.42 | 58.8% | 0.02 | 60.6% | <.001† | 60.5% | 0.17 |
| ABMS or AOA | 66.7% | 66.5% | 62.9% | 62.9% | 62.0% | |||||
| Years in Practice | ||||||||||
| <10 Years | 66.8% | 0.94 | 67.1% | 0.73 | 62.7% | 0.68 | 62.7% | 0.72 | 61.3% | 0.30 |
| 10–19 Years | 66.8% | 66.9% | 62.6% | 62.7% | 61.6% | |||||
| 20–29 Years | 66.7% | 66.6% | 62.4% | 62.8% | 61.9% | |||||
| 30–39 Years | 66.7% | 66.0% | 62.2% | 62.9% | 62.6% | |||||
| 40–49 Years | 66.7% | 65.8% | 62.1% | 62.9% | 62.9% | |||||
| ≥ 50 Years | 66.7% | 65.5% | 61.9% | 63.0% | 63.2% | |||||
| Medical School | ||||||||||
| International | 67.3% | 0.25 | 66.9% | 0.87 | 61.5% | 0.21 | 61.9% | 0.001† | 60.6% | 0.14 |
| Domestic | 66.6% | 66.6% | 62.7% | 63.0% | 62.0% | |||||
| Malpractice | ||||||||||
| No Malpractice Claims | 66.8% | 0.08 | 66.9% | 0.03 | 62.6% | 0.18 | 62.8% | 0.06 | 61.5% | 0.01 |
| 1+ Malpractice Claim | 65.0% | 60.9% | 61.2% | 61.9% | 62.9% | |||||
| Disciplinary Action | ||||||||||
| No Disciplinary Actions | 66.8% | 0.008† | 66.7% | 0.16 | 62.5% | 0.51 | 62.8% | 0.56 | 61.9% | 0.56 |
| 1+ Disciplinary Actions | 64.0% | 59.1% | 60.6% | 61.8% | 58.8% | |||||
| US News Research Rank | ||||||||||
| Attended a lower or unranked school | 66.8% | 0.82 | 66.8% | 0.79 | 62.4% | 0.06 | 62.8% | 0.19 | 62.2% | 0.003† |
| Attended a top 10 school | 66.6% | 65.7% | 65.4% | 62.2% | 59.1% | |||||
| US News Primary Care Rank | ||||||||||
| Attended a lower or unranked school | 67.0% | 0.07 | 66.7% | 0.84 | 62.6% | 0.42 | 62.7% | 0.33 | 61.9% | 0.34 |
| Attended a top 10 school | 65.3% | 65.7% | 61.8% | 63.3% | 60.9% | |||||
Statistically significant difference, allowing a maximum false discovery rate of 5% (critical P value = 0.0116).
DISCUSSION
Consumers are encouraged to use physician characteristics such as board certification and lack of paid malpractice claims as a signal for quality.1, 2 Yet in our study few individual physician characteristics are consistently associated with higher quality, and when present, these associations are small in magnitude and are generally not significant in a practical sense. If one just looks at the 3 physician characteristics that had an association with quality, the difference in overall composite performance between the average physician with the best combination of these characteristics (female, board certified, domestically trained), and the average physician with the worst combination (male, non-certified, internationally trained physician) is only 5.9%. Also, this is the average difference. Among physicians with the best combination there is a wide range of performance (48.8.5% to 75.3%, 5th to 95th percentile); this range is quite similar to the range of all physicians (48.2% to 74.9%). Thus, there is little evidence to suggest that a patient will consistently receive higher quality care by switching to a physician with these characteristics. Overall, the results highlight the need for externally available quality information for consumer use.
Despite the finding that physician characteristics are imprecise proxies for consumers to use in assessing quality, we did find some characteristics that were associated with higher performance. Board certification was associated with high performance scores at the overall level and with both acute and preventive care. We recognize this is an association and does not imply that board certification itself drives the difference between higher and lower quality physicians. However, this association does provide preliminary evidence suggesting that there may be some quality of care benefit to be derived from maintenance of certification programs or the inclusion of board certification activities as a requirement for maintenance of licensure.27 Further, while past studies have examined the relationship between board certification and quality in an assortment of specific clinical areas,15,16 this is the first to demonstrate a robust relationship between certification and clinical quality across a broad range of clinical conditions and types of care.
It is striking that we found no consistent association between number of malpractice claims or disciplinary actions and quality. Though malpractice claims have strong associations with measures of physician communication,28 physician communication style (and other physician attributes associated with malpractice claims) may have an inconsistent relationship to the process measures of quality that we investigated. Our results in this regard are similar to previous research showing little association between malpractice claims and physician quality as measured by health outcomes.[needs cite] In addition, the very low numbers of physicians with disciplinary actions against them by the board in our sample makes it difficult to detect any association.
In contrast to the previous literature, we did not find any associations between physicians’ years of experience and quality. There are several potential explanations for this difference. The previous systematic review by Choudhry and colleagues used a much broader definition to measure quality, including performance on theoretical evaluations such as written examinations or hypothetical clinical scenarios, guideline adherence for therapy or prevention, or health outcomes such as mortality; and included individual studies with narrow areas of clinical quality assessment.7 Our study utilized only process-based measures of quality of care across a broad range of clinical areas. Further, while the studies included in the systematic review assessing academic knowledge as a marker of quality all showed consistently negative associations between age and quality, results were somewhat more mixed when quality was measured by adherence to guidelines, a method more analogous to our own work. Lastly, while the majority of studies in the systematic review found a negative association between experience and quality, 21% of the studies in the review reported no effect, similar to the findings of our work.
Our study has limitations. The investigated physician characteristics are the major publicly available data on individual physicians that are easily accessible to consumers. However, we recognize that in the future, patients may have access to physician-level performance on some quality metrics. When available, these metrics may be different (and narrower in scope) than those utilized in this study. Further, though we utilized a broader range of clinical quality measures than any other study to our knowledge, the scope of the quality metrics is inherently limited. The RAND Quality Assessment Tools covered 22 conditions and included solely process-based measures. It is possible that there are stronger associations between physician characteristics and performance on quality measures that were not investigated, (e.g., measures of patient experience or mortality). Due to inherent limitations in medical claims, quality measurement using claims is less robust than quality measurement based in a medical records review. However, one key advantage of using claims is that it allowed us to assess quality of care for a large number of physicians.
Others have noted relationships between practice characteristics and quality measure performance, {Friedberg, 2009 #66, Pham, 2005 #39} but these practice characteristics were not available for the current analysis. Few physician practice characteristics are publicly reported by the Massachusetts BORIM, and their availability to patients who are choosing a physician is relatively limited. The question of whether generalists or specialists provide better care for specific conditions is not well addressed by our study, as we assessed the quality of care across an aggregated group of conditions, rather than on a condition by condition basis. This question has been investigated in other settings. {Smetana, 2007 #67}
Our study was limited to Massachusetts, a state with a high density of academic medical centers and higher overall quality of care than the national average.29 It is possible that in this setting of higher clinical quality, the effect of physician characteristics may be less important than it would be in a setting where the overall quality of care is lower.
In conclusion, we find that individual physician characteristics are poor proxies for performance on clinical quality measures and are not well suited for use as such by patients. Public reporting of individual physician quality data may provide the consumer with more valuable guidance when seeking providers of high-quality care.
Acknowledgments
The authors would like to thank Julie Lai and Scott Ashwood for their excellent programming on this work. We are grateful to Barbra Rabson and Jan Singer from the Massachusetts Health Quality Partners who facilitated obtaining access to the data sets used in this project.
Ms. Orler had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. The research was supported by a contract from the U.S. Department of Labor and a grant from the Commonwealth Fund. Dr Mehrotra’s salary was supported by a career development award (KL2 RR024154-03) from the National Center for Research Resources, a component of the National Institutes of Health. Ms. Orler was supported by a stipend from the University of Pittsburgh School of Medicine Dean’s Summer Research Program.
Footnotes
The research was supported by a contract from the U.S. Department of Labor and a grant from the Commonwealth Fund. Dr Mehrotra’s salary was supported by a career development award (KL2 RR024154-03) from the National Center for Research Resources, a component of the National Institutes of Health. Ms. Orler was supported by a stipend from the University of Pittsburgh School of Medicine Dean’s Summer Research Program. The authors have no potential conflicts of interest to disclose.
References
- 1.AARP. [Accessed October 2, 2009];Choosing Your Managed Care Doctors. 2006 November 7; http://www.aarp.org/health/insurance/articles/choosinghmo.html.
- 2.HeathGrades, Inc. [Accessed September 3, 2009];Find Out if Your Physician is a HealthGrades Recognized or Five-Star Doctor: Why is this Information Important to Me. 2009 http://www.healthgrades.com/consumer/index.cfm?fuseaction=modNW&modtype=shop&modact=recogs.
- 3.Boscarino JA, Adams RE. Public perceptions of quality care and provider profiling in New York: implications for improving quality care and public health. J Public Health Manag Pract. 2004 May–Jun;10(3):241–250. doi: 10.1097/00124784-200405000-00008. [DOI] [PubMed] [Google Scholar]
- 4.Ely JW, Dawson JD, Young PR, et al. Malpractice claims against family physicians - Are the best doctors sued more? Journal of Family Practice. 1999 Jan;48(1):23–30. [PubMed] [Google Scholar]
- 5.Khaliq AA, Dimassi H, Huang C-Y, Narine L, Smego JRA. Disciplinary action against physicians: Who is likely to get disciplined? The American Journal of Medicine. 2005;118(7):773–777. doi: 10.1016/j.amjmed.2005.01.051. [DOI] [PubMed] [Google Scholar]
- 6.Kohatsu ND, Gould D, Ross LK, Fox PJ. Characteristics associated with physician discipline: a case-control study. Arch Intern Med. 2004 Mar 22;164(6):653–658. doi: 10.1001/archinte.164.6.653. [DOI] [PubMed] [Google Scholar]
- 7.Choudhry NK, Fletcher RH, Soumerai SB. Systematic Review: The Relationship between Clinical Experience and Quality of Health Care. Ann Intern Med. 2005 February 15;142(4):260–273. doi: 10.7326/0003-4819-142-4-200502150-00008. [DOI] [PubMed] [Google Scholar]
- 8.Pham HH, Schrag D, Hargraves JL, Bach PB. Delivery of preventive services to older adults by primary care physicians. JAMA. 2005 Jul 27;294(4):473–481. doi: 10.1001/jama.294.4.473. [DOI] [PubMed] [Google Scholar]
- 9.Streja DA, Rabkin SW. Factors associated with implementation of preventive care measures in patients with diabetes mellitus. Arch Intern Med. 1999 Feb 8;159(3):294–302. doi: 10.1001/archinte.159.3.294. [DOI] [PubMed] [Google Scholar]
- 10.Kerfoot BP, Holmberg EF, Lawler EV, Krupat E, Conlin PR. Practitioner-Level Determinants of Inappropriate Prostate-Specific Antigen Screening. Arch Intern Med. 2007 July 9;167(13):1367–1372. doi: 10.1001/archinte.167.13.1367. [DOI] [PubMed] [Google Scholar]
- 11.Kim C, McEwen LN, Gerzoff RB, et al. Is physician gender associated with the quality of diabetes care? Diabetes Care. 2005 Jul;28(7):1594–1598. doi: 10.2337/diacare.28.7.1594. [DOI] [PubMed] [Google Scholar]
- 12.Berthold HK, Gouni-Berthold I, Bestehorn KP, Bohm M, Krone W. Physician gender is associated with the quality of type 2 diabetes care. J Intern Med. 2008 Oct;264(4):340–350. doi: 10.1111/j.1365-2796.2008.01967.x. [DOI] [PubMed] [Google Scholar]
- 13.Henderson JT, Weisman CS. Physician gender effects on preventive screening and counseling: an analysis of male and female patients’ health care experiences. Med Care. 2001 Dec;39(12):1281–1292. doi: 10.1097/00005650-200112000-00004. [DOI] [PubMed] [Google Scholar]
- 14.Flocke SA, Gilchrist V. Physician and patient gender concordance and the delivery of comprehensive clinical preventive services. Med Care. 2005 May;43(5):486–492. doi: 10.1097/01.mlr.0000160418.72625.1c. [DOI] [PubMed] [Google Scholar]
- 15.Sharp LK, Bashook PG, Lipsky MS, Horowitz SD, Miller SH. Specialty board certification and clinical outcomes: the missing link. Acad Med. 2002 Jun;77(6):534–542. doi: 10.1097/00001888-200206000-00011. [DOI] [PubMed] [Google Scholar]
- 16.Brennan TA, Horwitz RI, Duffy FD, Cassel CK, Goode LD, Lipner RS. The role of physician specialty board certification status in the quality movement. Jama-Journal of the American Medical Association. 2004 Sep;292(9):1038–1043. doi: 10.1001/jama.292.9.1038. [DOI] [PubMed] [Google Scholar]
- 17.Ko DT, Austin PC, Chan BTB, Tu JV. Quality of Care of International and Canadian Medical Graduates in Acute Myocardial Infarction. Arch Intern Med. 2005 February 28;165(4):458–463. doi: 10.1001/archinte.165.4.458. [DOI] [PubMed] [Google Scholar]
- 18.Rhee S-O, Lyons TF, Payne BC, Moskowitz SE. USMGs versus FMGs: Are There Performance Differences in the Ambulatory Care Setting? Medical Care. 1986;24(3):248–258. [PubMed] [Google Scholar]
- 19.Friedberg MW, Coltin KL, Pearson SD, et al. Does affiliation of physician groups with one another produce higher quality primary care? J Gen Intern Med. 2007 Oct;22(10):1385–1392. doi: 10.1007/s11606-007-0234-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.America’s Best Graduate Schools: Schools of Medicine. [Accessed June 22, 2008];US News and World Report. 2008 144(10):70. http://grad-schools.usnews.rankingsandreviews.com/best-graduate-schools/top-medical-schools. Published April 4–11, 2008. [PubMed]
- 21.Massachusetts Board of Registration in Medicine. Disciplinary & Other Public Board Actions. [Accessed June 20, 2008];A listing of Disciplinary Actions of the Massachusetts Board of Registration in Medicine from June 9, 1999 to June 1912, 2008. Available at: http://www.massmedboard.org/public/discipline.shtm.
- 22.McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003 Jun 26;348(26):2635–2645. doi: 10.1056/NEJMsa022615. [DOI] [PubMed] [Google Scholar]
- 23.Asch SM, Kerr EA, Keesey J, et al. Who is at greatest risk for receiving poor-quality health care? N Engl J Med. 2006 Mar 16;354(11):1147–1156. doi: 10.1056/NEJMsa044464. [DOI] [PubMed] [Google Scholar]
- 24.Reeves D, Campbell SM, Adams J, Shekelle PG, Kontopantelis E, Roland MO. Combining multiple indicators of clinical quality: an evaluation of different analytic approaches. Med Care. 2007 Jun;45(6):489–496. doi: 10.1097/MLR.0b013e31803bb479. [DOI] [PubMed] [Google Scholar]
- 25.Min LC, Wenger NS, Fung C, et al. Multimorbidity is associated with better quality of care among vulnerable elders. Med Care. 2007 Jun;45(6):480–488. doi: 10.1097/MLR.0b013e318030fff9. [DOI] [PubMed] [Google Scholar]
- 26.Benjamini Y, Hochberg Y. Controlling the False Discovery Rate - a Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society Series B-Methodological. 1995;57(1):289–300. [Google Scholar]
- 27.Federation of State Medical Boards. [Accessed October 3, 2009];An Analysis of the Impact of Implementation of Maintenance of Licensure Requirements. 2009 March; http://www.fsmb.org/pdf/mol_impact_analysis_report.pdf.
- 28.Levinson W, Roter DL, Mullooly JP, Dull VT, Frankel RM. Physician-patient communication. The relationship with malpractice claims among primary care physicians and surgeons. JAMA. 1997 Feb 19;277(7):553–559. doi: 10.1001/jama.277.7.553. [DOI] [PubMed] [Google Scholar]
- 29.Agency For Healthcare Research and Quality. [Accessed October 1, 2009];NHQR 2008 State Snapshots. 2009 June 5; http://statesnapshots.ahrq.gov/snaps08.